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FOREWORD 



This two-day seminar was arranged for the express purpose of training 
rehabilitation personnel in the procedures, theories, and aims of work 
evaluation. It was designed to discuss and tiy to find better ways of 
utilizing the knowledge we now have to meeet the demands for comprehen- 
sive or total rehabilitation of the handicapped. 

Meetings were held at the Pennsylvania Rehabilitation Center (PRC) 
in Johnstown. PRC is the largest cosqprehensive rehabilitation center in 
the United States, and lends itself favorably to studying the various 
vocational evaluation techniques employed, and their utilization in the 
placement and subsequent evaluation of clients in vocational training. 

This seminar was made possible by the Vocational Rehabilitation 
Administration of the Department of Health, Education and Welfare throuc^ 
a grant to the Research and Training Center in Vocational Rehabilitation as 
a cooperative effort with the University of Pittsburgh and the Pennsylvania 
Bureau of Vocational Rehabilitation. 

It is hoped that the formal presentations and discussions presented 
in these proceedings will provide insie^ts into the nature of new programs 
currently underway in selected VRA- supported projects and selected reha- 
bilitation facilities. 
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THE USE AND MISUSE 0? VOCATIONAL EVALUATION 
IN THE COUT'ISELING PROCESS 



Marten H. Breginan 



Coordinator cf Professional Services 
Vocatior*al Rehabilitation Center 
of Allegheny Coiinty 
Pi tt sbur gh , Perjisylvania 



In the past few years we have seen a marked, expansion 
in the use of the process of -assessing the individual both as 
to his work potentials and work ad.justment as these factors relate 
to the achievement of a specific vocational objective,” (l) which 
has come to be known as ’'vocational evaluation.” While this pro- 
cess is known by many names such as: Guidance Test Class, Work 

Sair 5 )le Testing, Reality Testing, Miniature Job Saii^le Testing, Pre- 
vocational or Vocational Evaluation, Testing Orientation and Work 
Evaluation in Rehabilitation (TOWER), there seems to be general 
agreement that the utilization of actual or simulated, industrial, 
service, or commercial operations are an effective instrument in 
measuring the disabled individuals potentials for "...integration 
or re -integration into society." (2) 

The field of rehabilitation has hailed vocational eval- 
uation as the panacea in overcom.ing all of the apparent limitations 
of standardized psychological tests. Psychologists have questioned 
its use because of the apparent lack of consideration of the classic 
requirements of reliability and validity in the construction and 
use of work samples. (Despite this objection it has the attraction 
of face validity) (3) Vocational evaluation has been used and ab- 
used, understood and misunderstcod. The literature has described 
how to organize a unit, how to develop work sairqpleso Evaluators 
have been trained in short-term courses, on-the-job, and now at 
least one university is planning uo offer a Master's degree pro- 
gram to educate evaluators, even though there are differences 
among facilities as to what the evaluator does, and what his train- 
in should be. In our agency we hire an individual as an evaluator 
because of the specialized, skill that he brings to the unit. For 
example, the clerical evaluator is a business major, the industrial 
service evaluator is a home economist. We believe that the eval- 
uator should, be a master of one area, and not a jack of all trades. 
Not withstanding all of the questions and apparent shortcomings, 
it has become evident that the process of vocational evaluation is 
here to stay. 
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Sp6is6r and Cchsn (L ' and the Association of Rehabiii” 
tation Centers (5’)> have both found that over ^0^ of the rehabili- 
tation facilities responding to a questionnaire concerning the 
services they provide indicated that they have vocational or pre- 
vocational evaluation programs. 

Of interest is the fact that 60% of these units have 
only been in operation for less than five years, with the majority 
having been started in the past two years o The passage of the 
Vocational Rehabilitatior. Airiendments of 196$ which encourages the 
development of vocational evaluation programs, particularly for 
the \ " disabled., will undoubtedly lead to a skyrocketing of 

the number of facilities which will utilize vocational evaluation 
as a basis for, or part of their programs ^ 

With the increasing use of vocational evaluation in re- 
habilitation, one might feel secure in conjecturing that there is 
a clear understanding of the results that can be expected from the 
use of this technique, how these results are achieved, who should 
be referred, and above all where it fits into the overall pattern 
of rehabilitation servises. However, we have not yet reached the 
point where there are clear cut answers to any of these questions. 

We are operating on the assumption that we have a unifying theory 
and approach to vocational evaluation, but in actuality there may 
be as mai^ systems of evaluation as facilities using them. Until 
we come to grips with the problem there will continue to be con- 
fusion on the part of facilities who are developing programs, coun- 
selors who are encouraged to use it, and. particularly the clients 
themselves . 

For a moment let me delve into the l^istory of vocational 
evaluation. There are several avenues which have led to our current 
understanding and use of this technique. The early apprenticeship 
system encouraged the prospective apprentice to try-out in several 
trades before selecting the one he wanted to fellow. In the same 
vein in World. War I the Port-Villex School (6) in Belgium felt 
that the only way the disabled soldier could be helped to select 
and appropriate trade training program was to have him try-out 
in several trade classes. On the basis of this try-out he could 
then select the course that he was interested in, and had the 
ability to congplete. Another avenue was the development of scien- 
tific job analysis, and the early work of Hansterberg who was one 
of the pioneers iix the field of psychological testing. He was 
confronted, with the problem of developing a test to select street 
car operators for the Boston Railway Company, After a great deal 
of thought he decided to build a model street c^ and. try-out 
prospective operators on this model. Psychologists have dropped 
this approach in favor of the current use of paper and pencil tests. 



Bregman 



A third basis of evaluation relates to the approach 
taken in the U.S, Army Hospitals in World War I. They developed 
the idea that the wounded soldier would be " . . .helped to find a 
healthy, optimistic, and creative attitude toward life," ( 7 ) 
throu^ involvement in work activities around the hospital. 

With the exception of the IDynwoody School in Cleveland 
the concept of vocational evaluation was not used formally untill 
1936. At that ti:;ie the Institute for the Crippled, and. Disabled 
brou^t together elements of their various training programs into 
a central unit called Guidance Test Class. This was done to provide 
clients with an opportunity to explore their potentials for, and. 
interest in the various trade classes, in some organized way. 

Since that time we have seen the approach broadened, not only in 
terms of the number of centers using evaluation, but also in terms 
of the scope and variety of work samples that are used. More im- 
portant however, the expansion has oc cured in terms of the pur- 
poses. Thus the three historical bases to evaluation, namely; 

.job try-out, work sample testing, and. therapeutic milieu are 
currently an intergral part of the process which is being used: 

1 . To observe behavior. 

2 . To determine potentials for training, and/or placement. 

3. To determine potentials for rehabilitation services . 

U. As a therapeutic milieu for effecting change in vo- 
cational self concept. 

5. As a diagnostic tool. 

6 . As a prognostic tool. 

Within the facilities themselves the evaluation may take 
place in a separate unit, with its own staff. It may be part of 
a workshop, it may be part of a total constellation of con5>rehensive 
rehabilitation services. In some programs clients are paid and 
not in others. Our own approach is to have a separate unit which, 
however, is closely integrated with our workshop. We conceive 
that the workshop focuses on work adjustment and. evaluation 
focuses on the diagnosis of vocational handicaps as well as apti- 
tudes, interest and abilities. Clients are not paid, in evaluation 
but are in the workshops. 

With all of the present dialogue and. claims concerning 
evaluation, is it any wonder that the counselor may be troubled as 
to how it fits into his frame of reference when he is encouraged 
to refer his clients for vocational evaluation. (I am referring 
both to the BVR counselor and the facility counselor.) He is not 
quite sure of who can benefit, and why, to what extent can he rely 
on the data provided, him after evaluation. Let us try to resolve 
some of this d LLema by suggesting that vocational evaluation can 
be 11 that we have said it is, and that the problem relates to 
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how it is used^ rather than what it iSo Use not in terms of the 
rehabilitation process j, but rather in a somewhat more narrow sense 
in terms cf the counseling process itself o 

Why counseling? If ws examine counseling it can be 
seen that it is a process of the formulation of questions or hypothesis 
concerning the client and his behaviors^ with the client providing 
the answers which can then act as the framework for further ques- 
tions with the ultimate objective to ooohelp individuals toward 
overcoming obstacles to their personal growth, wherever these may 
be encountered and toward achieving optimum development of their 
personal resources! (8) It is only the client who can provide 
the questions and answers, but it is the counselor idio must deter- 
mine what are the real questions « For exair 5 >le, if a client says, 

”I can®t find a job,” it may be because he hasn^t looked, or be- 
cause he is applying for jobs beyond his capabilities, or even 
that he hates his mother « The counselor must explore each of 
these hypotheses with his client, to determine which are correct 
and which are incorrect u These answers then provide the basis for 
further eaqploration until the goals for that client have been a- 
chievedo In the course of these explorations the counselor may 
use a variety of tools to aid himo For example, if a hi^ school 
graduate asks the counselor what career he should follow, the 
counseling process is generally initiated by asking if he has the 
potential for higiher education o The counselor examines transcripts, 
test scores, may administer additional tests, observes behavior to 
provide answers to that first question.. If the information sub- 
stantiates the hypothesis then he can complete college, the client 
can then be helped, to explore occupations which require college 
level training ^ However there may be other factors, such as per- 
sonality problems, which ir.terfere with the attainment cf goals 
consistent with the clients intellectual ability . We may then ex- 
plore these questions with continued interviews, personality tests, 
or psychiatric interx’iewso As we continue this process the client, 
either verbally or through our diagnostic tools, tells the counselor 
that he is or is not on the right track o When he is, that avenue 
is followed until a logical conclusion is reached. Logical not to 
the counselor, but to the client. After all, he is the one who 
must act. Thus ’’classical counseling” becomes a verbal eaqploration 
with the use of diagrAOStic instruments as an adjunct to the pro- 
cess, to provide a foundation for future action. Obviously this 
is an oversimplification of a coir^»lex process. However, if we 
examine vocational evaluation from this frame of reference it can 
be viewed as a tool for the e^qplorations of the counselor's, and 
client's, questions concerning the handicaps to, as well as the 
assets for, vocational planning. 

When a counselor sees his client for the first time, he 
is generally confronted with the statement, ”I want a job.” The 
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first step is tc initiate diagnostic procedures to detennine (!l) 
whether the client has a disability, 2 what it is, and ‘3 is it 
correctable The counselor must then determne.the extent to which 
these disabilities handicap the individual in the attainment of a 
vocational objective, and h what can the client do> For many clients 
the answers to number 3 and It can be derived throu^ interviewing, 
checking of references, cr psychological testing. However, in maQy 
instances the counselor is confronted with -he problem that he has 
all this informaticn, but for some reason the client does not 
accept the interpretation. For exan 5 )le, the machinist who has lost 
a leg, and now has a well fitting prothesis. The preliminary data 
reveals that he has the potentials to return to his former oc- 
cupation, but no matter how much the counselor tries to convince 
the client, he says, you know I can*'t do that work anymore. In 
this instance the counselor can hypothesize that the only way the 
client will develop an understanding that he can do his former 
job is to place him in an evaluation unit so that he can see for 
himself that he can do the work. Ideally, the client will then be 
able to act on this: information, and return to his former enploy- 
ment. If this does not occur, then other questions must be raised. 
Perhaps he really doesnH want to be a machinist, or any one of a 
multitude of hypothesis which can be raised. In this case the e- 
valuation unit serves to raise additional questions which require 
further e^qploration. But why is this approach required? Wly cai *t 
the client act upon the information obtained through psychological 
tests? Reissman, (9) in his formulation of learning styles, sug- 
gests that we can only communicate to the individual throu^ his 
characteristic learning style. These briefly are oral, verbal, 
and physical. Ifeny of our clients are physical learners and can 
only perceive their vccational strengths and weaknesses throu^ 
handling an object. The fact that people who score hi^ on the 
Purdue Pegboard make excellent butter wrappers, has no meaning to 
the physical learner when interpreted verbally. 

In another case the counselor can be confronted with the 
situation in which classical diagnostic procedures are ineffective 
in determining the individuals pattern of interests, abilities, 
and aptitudes. For example, if the client is unable to read, what 
patterns can be measured on standardized tests? The only way we 
can develop an understanding cf his abilities is to have him go 
through a series of graded vocational activities. Thus the hy- 
pothesis, what can this client do, can be explored throu^ the use 
of the process of vocational evaluation. For the ’’retarded", or 
cerebral palsied, or mental patient, the vocational evaluation 
unit provides the opportunity not only to explore a number of vo» 
cational areas, but also it is becoming increasingly iitqaortant as 
a technique for the determination of problems which may be inter • 
fering with the assessment cf vccational potential. Thus as a tool 
of tbe counselor the evaluation unit is ideally glared to exploring 
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a number of hypotheses and more often than not, the program raises 
further questions concerning the services required a client. 

What you must ke^ In mind Is that vocational evalua- 
tion can provide the Information, but the counselor must use It. 

It does not do anything by Itself. You must provide the questions, 
and the frame of reference. Questions such as: 

1. Can this client be trained to be an auto mechanic? 

2. Can this client be placed as a janitor? 

3. What can this client do? 

U* What are the factors idilch may be Interfering with 
his vocational potentials? 

and the iqyrlad of other questions that you have about your client. 
Incidentally, In our own agency one of the frustrations of the 
evaluator Is when the counselor says evaluate this client, period. 
He wants to know where to begin, and more Important idqr Is he be- 
ing evaluated. This then provided the starting point for the 
process. 



In our ofm program we have spent many hours considering 
how we can utilize the evaluation unit most effectively. At first 
we concentrated on the assessment of vocational 2 g>tltudes, Inter- 
ests, and abilities, to provide Information which formed the basis 
of our counseling. In recrat years we have come to realize that 
evaluation can provide a much broader diagnosis of the client's 
handicaps. It was only last ve^k that we discussed a client niho 
was spending all of his time roaming around the unit, and talking 
to others, to the point idiere his behavior was Interfering with our 
ability to develop a picture of his aptitudes. The staff decided 
that we must d*^velop a plan to control this behavior, and once 
this objective has been acconqpllshed we can then continue the e- 
valuatlon. In this Instance the vocational evaluation had eval- 
uated this client's handicap, and provided the basis for the next 
step. In another case a young man was referred, and on the basis 
of preliminary screening. Including Interviews and psychological 
tests. It was determined that he had the potential for college 
level work. However, his aspiration was to be an auto -mechanics 
area that he would change his mind. As It happened, he found that 
area very difficult, and Is presently In college. Thus, the coun- 
selor's hypothesis was substantiated, and he could then help the 
client achieve a "realistic" goal. 

One of the major questions confron!/lng the counselor Is 
^o should he refer for evaluation. As suggos'^ed earlier. It Is 
a process Ideally geared to the physical learner. I am not com- 
pletely sure, at this point, how we can determine niho Is the 
physical learner, but In general he Is the person who has found 
It difficult to learn academic subjects In school, or a person who 
has been engaged In manual or service occupations. 
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Another guideline is suggested by Gorthy. (10) He states 
that vocational evaluation is ideally suited for those individuals 

” , . owhcse vocational fitness or lack 
thereof are not readily apparent . » . 
the vocational evaluation will re- 
veal the extent to which these handi- 
capped individuals are capable of 
obtaining either limited or ultimate 
objectives . " 

Not withstanding these general guidelines^ the question 
has been asked by our own staff, who can we serve? In our own ex- 
ploration of that problem the answer is not who, but rather what 
problems can we deal witho The evaluation program can identify 
the problems, and from that we can then determine idiether we have 
the esqpertise to solve the problem <> Let me clarify. Consider the 
young man I alluded to earlier whose problem was the roaming and 
over -verbalization, which were interfering with the evaluation 
process. There could be several reasons for this behavior, (l) 
Perhaps he had a medical condition which manifests itself in over- 
verbalization. In this instance we have to ask the question, can 
it be controlled through medical intervention? However, the key 
to this young man’s problem was that he had spent five years in 
his home looking at four walls, talking only to himself and to 
the radio, and occasionally his mother. Thus, it became evident 
that if we were to evaluate his vocational potentials we had to 
eliminate this primary problem. The staff then hypothesized that 
he needed a controlled environment working on a one -step sinqple 
activity where we could begin to apply pressures to eliminate over- 
verbalization in the work setting. He also needed to have an 
opportunity to socialize with others, in the non -work setting. To 
acconqplish this objective he was placed in our workshop. We then 
hypothesized that once the over-ve baiization is eliminated, we 
can move him back to the evaluauicn unit for determination of his 
vocational potentials. In this sense the unit provides the diag- 
nosis of the key problem, the staff provides the reasons idyr, and 
the workshop prov *s the place idiere we can put into practice the 
plan that we developed-. In this instance, the evaluation unit 
played a dual role, for the determination of the problem, and then 
for the determination of the client’s aptitudes arid ability. But 
all of these do not happen by themselves . There must be active 
participation of the counselor -s part, even if only to supervise 
the plan. 



One potential danger in the use of vocational evaluation 
is for the counselor to ejqpect more than the unit can give. I 
have described what I consider to be the many possibilities in 
evaluation. Certainly I have used our own prograpi as a model. 
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wherein we have tried tc prc lie a bread base of ur. derstanding of 
the potentials of si;h a pregrair: F:t not all. evaluation prograv,s 

can answer all the querticr: yz. rr.ay have^ not even onrSo You 
must know the questicts which ‘car. be answered^ and use the varic;.? 
pregrams at your dispcral selectively 

In summary, the ni..se cf vocational evaluation in the 
counseling process cores from a iacK of understanding of where the 
counseicr fits into the prccecs He must formulate questions He 
cannot ejqcect the eval-.ator tc preside answers in a vacuum o He 
cannot expect evaluation by itself tc change behavior, no mere than 
we can ejqpect psychclcgical tests by themselves to change behavior^ 
or develop understanding in the clients The counselor must inter- 
pret to the client why he is going through all this rigamarole 
just to get a job. He must interpret the findings to the client. 

I wonder how anyone of us would feel if we were denq>ed into a pro- 
gram for eight or ten weeks, or even more, without knowing why, 
or knowing how we were doing. We would resist, and perhaps be 
unmotivated o The counselor must develop a treatment plan on the 
basis of the information he recievedo ^y treatment, I mean train- 
ing, placement, behavior modification, not psychotherapy « It is 
a tool to be used, but net abused^ 
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PREDICTIM5 VOCATIONAL ADJUSTMENT IN THE MEMTALLT RETARDED 

Henry V . Cobb, Ph. D. 

Chairman, Department of Psychology 
University of South Dakota 



particular interest in rehabilitation has been with 
the mentally retarded both throu^ research and through the at- 
ten^t to develop programs of education and rehabilitation to 
enable the retarded to take their place in society which has hither 
to extensively rejected them. I gather that your interests as vo- 
cational counselors are quite varied, that you deal with various 
types of disability categories and in a variety of different ways. 
So, while I direct my comments particularly to the subject of the 
mentally retarded, I hope some of them will be applicable to your 
particular concerns, because many basic questions will go across 
the board. 



The particular field I have been concerned with for the 
last several years grew out of an interest in VRA to find out what 
the outcomes have been of some research projects idiich they had 
sponsored. There are a number of such projects designed to deter- 
mine how we can, and how well we can, predict the vocational ad- 
justment of the mentally retarded in adult life. How far cot we 
see ahead? How much can we see ahead? What sort of determinations 
be made j n order to guide the tr ainin g and rehabilitative pro- 
cess to its most effective conclusion? The problem of predictive 
evaluation, of course, runs all throu^ the rehabilitation program. 
How do you know idiat to do unless you have some idea of the prob- 
able outcome? A good deal has been done in this connection in re- 
lation to various categories of disability. 

Now, I am not going to be technical here; I am going to 
deal rather with concepts and problems because I must be frank to 
say that so far the investigations I have been making lead rather 
to more questions than to final answers. I think we still have a 
long way to go before we really have nice, neat, predictive pro- 
cedures that will give us firm answers as to irtiat lies ahead for 
the individual client. I think we have to clarify idiat we are try- 
ing to do, and to see clearly what some of the problems are before 
we rely too heavily on premature conclusions. 

Let us start out with just a brief consideration of what 
the objectives of the vocational counselor are. What is the voca- 
tional counselor trying to do? In this I want first of all to make 
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a rather obvious contrast: the contrast between the process of 
personnel selection and the process of adjustment counseling. I 
am sure I don't really need to make this contrast to you because 
as counselors I think you understand this difference already. In 
personnel selection^ we start off with a particular function^ a 
particular Job^ or a particular operation^ or a particular business^ 
idiatever It may be^ and the purpose Is to fill slots. We have so 
many places for so many people^ and these people are supposed to 
perform certain functl<ms and we want them to perform these func- 
tions successfully. Our Job In personnel selection^ then^ Is to 
pick the people idio will fill these slots with the maximum prob- 
ability of success 4 and the minimum probability of failure. We 
can use the Information that Is available to us In determining 
where the probability of success lies and where the probability of 
failure lies. We will establish cutting points in terms of test 
scores or whatever other predictors we are using In order to select 
that group of people idio will least likely let us down and will 
most likely be successful. That Is the nature of personnel selec- 
tion and It Is a fine art^ used extensively In business and Industry 
idiere the cost of failure Is part of the production cost^ and thoy 
want to keep that at a mi nriimim . On the other hand^ adjustment coun- 
seling begins with the person. Here is a person for idio we want 
to provide services tdilcdi will maxiiiiize his function as a member 
of society^ as a productive part of the communtiy^ to live aQ full 
and whole a life as possible. Starting with the Individual^ the 
aim Is to work with him under the best possible conditions. For 
this we want to predict the probable outcomes of using various 
treatment procedures^ and to find that combinatlcm of procedures 
idilch will maximize his probable success and minimize his probable 
failure. The criterion here is the person^ whereas in personnel 
selection the criterion was the security of the enterprise or for 
idilch you are finding people. Here we are finding or doing some- 
thing for the person; there we were doing something for the parti- 
cular slot In idilch the person Is fitted. 

Obviously^ these are two quite different aims and obviously^ 
your concern Is with the second. Yet^ I think It Is necessary to 
point out that often we do confuse the two. Often we have primarily 
in mind the situation within which we are operating as counselors^ 
and are concerned with the criteria by idilch our success Is to be 
measured. Often we are confused as to the real values that we are 
ftinring at. I am sure that in Pennsylvania this Is long since past^ 
but in some places the success of a vocatl(xial counselor has been 
measured in the number of people that he has successfully "rehabil- 
itated". You have seen that phraseology^ I am sure^ time after 
time. If that Is the measure of a successful counselor^ then It 
Is up to him to obtain a maximum score. You get a maximum score 
idien you have the most successes with the fewest failures^ and you 
thereby inmediately fall Into the role of personnel selection. 

You are selection people who will succeed for you^ and avoiding 
people nAio will fall for you. This Is a natural human tendency to 
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seek the rewards where they may be found, but often we fall into 
confusion as to what we are really trying to do. 

Sometimes we are confronted with this word **fea5ibility**. 
This is the question with respect to any client: is it feasible to 
provide training and rehabilitative services for him? How, as soon 
as you try to answer the question of feasibility, you are talking 
about probability. You are talking about his possible success or 
failure in terms of some criterion. If you take the broad criterion 
of employability, then you are asking yourself, will this person 
be successful in terms of employability if we take him under our 
wing and provide him with training services? You are making a 
prediction and if you are not careful you may predict out some in- 
dividuals who may, in fact, succeed, and you may be inclined to 
regard as feasible only a limited proportion of your personnel. 

I am suggesting here, you see, that we keep our aims 
clear. If we are seriously concerned with adjustment counseling 
then idien we talk about feasibility, we will not ask, "is it 
feasible to put the client in this particular training process which 
we have going?", but we will ask, rather, "what is feasible for 
this individual, and what are the alternatives among which we 
can select a program, and a course of training?" Again, the role 
of the vocational counselor is to serve the individual person, 
not merely to select the most probable successes from a range of 
candidates . 

Assuming, there, that what we want is information which 
will enable us to make the best possible program arrangement for 
the individual client, how do we go about obtaining it? The first 
f.h-i ng to recognize is that predictive information is needed not 
at one time only, but at many successive times along the rehab- 
ilitation route, and this information takes many forms. We have 
already mentioned the question of feasibility. This is part of 
the Intake procedure. When you are first considering a client, 
you consider in irtiat terms is it possible to serve this client. 
Feasibility determination is then one sort of evaluation, based 
on whatever information is available at the outset. But then one 
to narrow this down and evaluate the client in terms of the 
type of training that would be moot productive. Where do you 
start out with this person? at what level? in irtiat sort of set- 
ting? in what sort of training? what kind of treatment? The 
evaluation then is based on the particular description of the 
Individuals and his characteristics obtainable at Intake. 

But then as training is instituted, and treatment is 
provided, one makes evaluations along the way . How do you move 
this person through a course of training? What determines when 
he is ready for the next step? What determines what step follows 
from the step that he is already in? What determines idiether you 
retain him in training or whether you terminate the nrocess? 
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What determines when he i3 ready for those stages of training when 
we place him in some kind of en^JXoyment? 



When do you regard the case closed? Closure of a case 
involves just as much prediction as opening a case^ because you 
are predicting that this person is now equipped to function under 
normally operating social arrangements, so that your services are 
no longer required » When do you reach that point, and what criteria 
do you use for making that kind of judgment? 

In arriving at the decisions along the way there are 
some particular areas about which we are concerned and about idiich 
we make predictions o Occupational selection, of course, is one* 

What sorts of things can this individual be equipped to do? What 
kind of role can we eapect him to play in the economic life in so- 
ciety? We are also concerned with the other aspects of the person 
which surround the fulfilling of occupational roles* One works, 
of course, only during certain portions on one's living time* The 
rest of the time one is doing all sorts of other things which have 
profound bearing on how effectively one works* Althou^ we call 
ourselves vocational rehabilitation counselors, we cannot isolate 
vocation as work from the rest of the process of living* So one 
makes predictive evaluations of all these things as well* 

Evaluation has many points of attack but in every case 
it involves using presently obtainable information in order to 
make some kind of prediction as a guide to action* What, then, 
are the requirements for effective predictive evaluation? 

Basically two major kinds of information are needed* On the one 
hand, we have to know what it is that we are trying to predict 
and how this can be measuredo What is the outcome or the end 
result that we ai*e looking for? In other words, what are the 
criteria in terms of which we make a prediction? It seems s^le, 
but in fact the selection of criteria is a tremendously compli- 
cated business o 

When we talk about successful adjustment, what are we 
really talking about? Often we narrow it down and define it very 
specificallyc We say, the criterion of successful adjustment is 
the ability to hold a jobc Well, hold a job? Hold a job how 
long? Hold a job under what sorts of working conditions? Hold 
a job at what level of ccn^lexLty? Even holding a job is a com- 
plex matter o But we find as we explore the criteria that there 
are many, many others I will mention a little later a particular 
study which analyzed the judgments made by ten experts in the field 
of mental retardation c The investor came up with a total of lUl 
specific criteria of success, which then had. to be factor analyzed 
in order to be handled in a meaningful way. 
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would want the o-.^orr^.e of the ptrior to oe .jud^,-c3., On. can org^ 

ize these criteria in all sort? of wayt . 1 think they general y 

fall under about five headings 

1 Personal factor_s These are factors which are^^ 
coiOTonly subsumed nnaer ths rather vague term "!elf concept, or 
"S? -aliu stment . " Thi s in itself i rather genera.^thing and 
needs to be broken dcwn intc. parts rcr ey«ctple, Jt.^n^ - - p 
sonal factors are the image the has of Mmseif , r“® 

well-adjusted person it one who dficntc-c i.ir-eif .ec^j±. 7 

cr ^n some degree ”ldeallctUa'/'.y’' in terr? of what ne hopes to 
obtain. The badly adjusted person has very mixed ^ * 

whc he is, what he is like, what he is good for. what he can do, 
and what he really waists. Along with the descriptive -'®y-image 
there is also self-esteem, the value thet one places upon oneself, 
the degree of self -ac ceptance cr -l^rejection, ^.d the 
ment or discontent with being a-: one txndr" himi^elf. ihe self- 
concept, there, includes all the a.trlbutes of the person he 
sees himself and as he evaluates kircseli with ail his capabiliti 
and limitations, and the '-’.ay in which he identifies himself as M 
individual psrsor.. Another part of the 

in the goals a rerson ka= and how he airires .0 tnem. The aspir- 
ation levels of the person which ocnsTttve a part o. hiS seif- 
system include his respcnGes tc ?uc.'.05j arc .aiiure^ h^c abil y 
to meet frustration, his sroticnai stabiJi.n^ general Aoove 
all his control and defense sy items determine how n.e m^ageo his ^ 
personal e:dstance . The e-xr,er t to x’hich he has control of his 
resources so that he can oUTito them, the ewtei-t to w.iich he 
employs psychological defense tysteiT^s in oroer tc manage ^rjs ra 
tion or to defend himoelf agai-s<- hurt, ai.. cf these ^ all in the 
area of persona.1 factors that a. inrolveo in vihat we cal^ a 
successful life. The textbooks end treatises on personality 
xill provide you., I am sure, with thousands of nctions that you 
can develop in reiaticn to the persona-: aspect of the individual, 
I am, also sure that if wc a.sk yoc x^hat you mean by a successful 
rehabilitation you would come up with many terms that would fall 
into this particular group. 

2 There IS ancther gro,.p cf terms that we would sub- 
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surrounding world, being located in space and time £«o that one 
moves effectively in these reference points. VJhat can a person do 
for himself that people would ordinarily do for themselves? Ade- 
quate adjustment again can be measured, by these criteria. 

3 . The third category would be in i nterpersonal re- 
lationships. The person is not an isolate. He is a person among 
other persons, and. we are here concerned with the ways in wJiich 
the person interacts with others. Family ties, family re3.ation- 
ships; friendships, affectional interchanges between one person 
and another, and the stability of such affectional behavior; 
social -sexual relationships, marriage, children; these are some of 
the principal ways in which people interact as persons with one 
another. This constitutes an area of experience which I know we 
all see as of vital Importance. Where this is disturbed, almost 
everything else is disturbed. So this would be a third major 
basket into which we could put the criteria for successful adjustment. 

U. In the fourth place another set of social relation- 
ships. But these we might call *» impersonal social transactions ." 

A great deal of our normal everyday lives involves dealing with 
people, not as individuals, as in terms of the roles that they 
perform. We go to the store and we deeil with the clerk. We come 
to the cashier at the end of the cafeteria line. We engage in 
these transactions which constitute the network or fiber of social 
bdiavLor in terms of idiicb we must live. This provides the 
security for the fundamental operations which sustain life in 
society and culture in general. It includes the commercial trans- 
actions of everyday life, transport ational transactions, legal 
transactions. Obeying the law and conforming to the legal re- 
quirements of society may become very con^licated, it is invol- 
ving the patterns of behavior, the mores and. custojus of the 
community in terms of such basic transactions as prope rty re- 
lationships, taxation, life insurance, civic activity of all sorts, 
and voting. One wants to know the extent to which an individual 
client is going to be able to do these things. 

Then in the fifth category we have productive be- 
havior. This would Include Job skills of all kinds, perceptual 
motor tasks, the verbal tasks which are involved in many Jobs, 
the Integrated behavior of operations, machine and tool management, 
all those things which are involved in the production of goods 
and services. The occupational variables which determine the 
production contrast of the clients potential economic employment. 

As I have looked at the wide range of possible criteria 
for successful adult adjustment, it has seemed to me that they 
all fall into these five major categories. Taking altogether, 
what we are asking is, what goes into the making of a full and 
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adequate life: the personal self -related elements of experience, 
the self management of the ordinar7 things, the interpersonal ties 
that we have, the ln9>ersonal social relationships, the productive 
behavior in which we use our capabilities to produce goods and 
services* These categories cannot be taken singly, in isolation 
from one another, but form a network. 

As a matter of fact, if we start talking about employ- 
ment as a criterion, we immediately find ourselves drawing in ele- 
ments from all five major categories. If we want to predict em- 
ployability, we find ourselves having to predict all these other 
things which go above with it. 

Let us now talk about another aspect of the evaluative 
process which also con9>licates things* 

There are so many situational variables which affect the 
life history and rehabilitation of a client that we cannot develop 
predictive norms which will work everywhere for all people in the 
same way. Vlhat produces successful functioning in one setting 
does not necessarily produce successful functioning in another 
setting* Maybe this would be a little clearer if I indicate some 
of the kinds of variables that I have in mind here. Geography, 
for exanqple, makes a difference in the way we look at possibilities 
for people. Here you are set in the middle of the mountains. 

Does this make any difference in the kinds of treatment and. train- 
ing and the kinds of goals, the kinds of objectives that would be 
appropriate to people you serve in this particular locality? Com- 
pare that with the great plains out idiere I live. It*s a very 
different setting. The climatic conditions are different. As you 
go farther south, climatic conditions are very different again. 

You have to deal with people in terms of geographic and climatic 
variables* You cannot just assume that people in one place will 
operate as do people in other places. Demographic variables of all 
kinds make a difference, the distinction between rural and urban 
for example, because people in cities behave differently in many 
respects from people living in the country. Population density, 
the size of the community in which people are living and operating, 
ethnic variables, patterns of culture, which generate not only 
different ways of behaving, but different value systems, different 
goals, different objectives. Economic variables which may shift 
and change from time to time make a great difference in ^at we 
can predict. We have been living through a period of prosperity 
in which employment opportunities are high. Should we go into a 
period of economic depression, what difference would this make in 
the way in which our rehabiltative efforts work out? The nature 
of the labor force at any time, the occupational patterns of the 
community of the region, all these are situational variables which 
affect our ability to predict what the outcome will be in the in- 
dividual case. 
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We have talked about criterion variables and situational 
variables^ non let us go over to another side of this prediction 
criterion relationship. What sorts of information can we make 
use of in order to predict the outcome? What can we know about an 
individual person as we look at him^ as we test him^ as we measure 
him in all sorts of ways that will give us a basis for looking ahead? 
These are the predictors, or, in research language, what we call 
the "independent variables," the casual factors; which bear some 
relationship to outcoiaes. 

As we look at the possible things that we might use as 
predictive measures, again they seem to fall into a number of 
categories. 



1. Background history . This involves many things re- 
lating to the individual. We always take a case history of the 
client. We want to know when and where he was bom, the charac- 
teristics of his ethnic background, his medical history, his 
educational history; our search broadens out over a great many 
subcategories. The background history factors are, in a way, 
settled. These are the things that happened to this person in 
the course of his life, but they are not always easy to discover. 

2. Intellectual competence fac^^. This is the way 
in which the individual functions intellectually, positively. 

Whatever ijiuelligence means, we want to find out idiat or how the 
individual measures in this area. Intelligence itself is not a 
sin|)le Udjog. It has many phases and aspects and we want to get 
a broad picture of the intellectual functions of the person. The 
I.Q. is probably the easiest but most misleading and least useful 
way to describe the person's intelligence as an adult. 

3. Physical and plarsiolo gic a] factors . This involves 
questions of physique, bo^ly structure, the nature and func- 
tioning of the various organs and systems of the body, the 
physiological mechanisms by idiich the bodily processes go on. 

It is important to know, ^or example, if the individual has a higher 
low metabolic rate, vital capacity, endocrine balance. Theses t hin gs, 
many of than, can be inqportant determiners of outcomes. 

li. Perceptual naotor factors . These are sometimes mixed 
with the intellectual competence factors, but are basically separable. 
They constitute the ways in which an individual can use his sen- 
sory equipment, and the integrated functioning in which he responds 
raotorlcally, in gross-motor fine-motor, language or speech 
funcions. 



5. Social behavior factors. These measure the in- 
dividual in the relationships he has to other people, using all the 
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evidence which is available to us for this kina cf behavior. Per- 
sonality factors can be colored at any one time in order to pre- 
dict personality structures and consequent behavior at a later 
time. We can ejqslore the present work habits and learned skills^ 
that is, the behavior patterns which he now employs in working 
situations <, 

These are general categories, then, within idiich we 
might try to acquire evidence which would enable us to make pre- 
dictive evaluations. These are the kinds of things idiich we ex- 
plore at various stages along the way in dealing with a problem 
of rehabilitation. The problem, then, which confronts the re- 
searcher in ejqsloring these things is ejqsressed goes out in a 
question which I put to you counselors: just how good is the 

evidence that you can collect for the things that ^iTreally want 
to know? There is not a rehabilitation center in the country that 
does not use some kinds of procedures of this sort. You have bat- 
teries of tests. You have an intake in history. You have physical 
e xam i n ations. You do all these things routinely. The question is, 
what good is all this doing? What does it enable you to predict 
with respect to the life course of the individual person? 

Let me make some comments here on the nature of the re- 
search process as against the practical work which you people are 
doing. Research in its best form is hi^ily controlled and tech- 
nical. It tries to get at underlying relationships. In doing this, 
it has to detach itself from the actual working situation in which 
people have practical concerns. However, as the researcher does 
his work, his findings have to be translated into practical forms 
in which they can be put to use. Often, however, there is a dis- 
crepancy between what the researcher can say and what the practi- 
tioner can be sure of, because what comes out of the research pro- 
cess may be very hi^ly refined; it may be isolated and detached 
from all things that are going on in the work -a -day world. This 
is true in any science. Galileo, for example, established the 
laws for falling bodies whereby certain predictions could be made 
how bodies would fall in a perfect vacuum, but then if you try 
dropping things throug^i the air they do not work that way because 
of the intervening effects of the atmosphere. Similarly, many 
of the results of our research may seem a little remote from the 
things that you are doing, or you may optimistically take scrnie 
of these findings and ejqpect them to work out neatly in everyday 
applications. It may not be true. For example, suppose we find 
that there is a predictive relationship between, let’s say, the 
amount of parental pressure put on a youngster and his aspiration 
level as he gets into a training situation. This might mean sinq>ly 
that if we take a large population of people and measure them in 
terms of the amount of parental pressure that is being applied, 
and then we measure these same people again on another occasion in 
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terms of their motivational structure we mi^t find a connection 
that is statistically better than chance. This may hold^ however^ 
only when we are looking at a whole population in which there is 
a certain trend or tendency for outcome to pile up on one side 
corresponding to information that we have gotten on the other side. 

But you cannot assume that every client that comes your way who 
mas subjected to parental pressures is going to show this partic- 
ular outcome because you do not have under control all the other 
variables that nL^t make a difference. You do not know whether 
the individual case is going to fall within the generalization we 
have made about the population. 

What I propose to do now is to review briefly a selective 
saiiQ>le of some recent research. Let me give you five sanqple pro- 
jects that have been undertaken in the area of the predictive eval- 
uation of the mentally retarded for adult adjustment. These will 
represent quite a variety of projects because I want to illustrate 
some specific points. 

The first one was undertaken Joseph Pamicky at the 
Johnstone Training School in Bordentown^ New Jersey. This is a 
facility for mentally retarded adolescent and the young adult 
males. The particular project undertook to see how well we can 
predict the clients progress throu^ the various phases of train- 
ing and into the final phase of enq^lpyrnent from testing at intake. 

There were five phases in the evaluation and training 
procedure. The first phase is that of prevocational evaluation 
which takes a period of ei^t weeks. They use a number of tech- 
niques. One consists in a unit evaluation on UO sanqple tasks or 
work sanq)les which are drawn frcxn typical operations and standardized. 
Another is a field evaluation^ as they call it^ where the person 
works in different occupational work areas in the institution and 
is evaluated for performance by the supervisors. A third kind of 
evaluation is the psychological test battery which includes motor 
tests ^ personality and temperament tests ^ and intelligence tests 
of a standard variety. They have^ there^ three kinds of information: 
work sample information^ on the job information^ and psychological 
test information. The question then was^ how much can we predict 
from this initial Information and what will follow in succeeding 
phases. Phase two which followed after ei^t weeks involved half- 
time training in work areas and half-time in academic types of 
training. In working half-time in the work areas ^ the student 
was rotated throu^ a variety of such areas. In phase three the 
more academic type of training was now terminated and the students 
were full-time in the work training areas. In phase four^ while 
they continue to reside in the school^ the trainees were employed 
in the comnunity in various job settings. At phase five^ they 
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left the institution to reside in the community where they were also 
employed. This represented the final phase of separation from the 
institutional setting^ but they were followed up to learn what 
would happen. 

The outcome in general was this: the researchers found 

that the predictions were best when they were directed toward the 
second phase. That is^ the intake phase ^ the prevocational eval- 
uation^ which enabled them to predict fairly well how these young- 
sters would do in the next phase ^ where they were half-time in the 
training areas. But as one looked farther ahead to the succeeding 
phases it was found that the predictive value of the intake infor- 
mation diminished. At phase four^ they found practically zero cor- 
relation between the intake testing and performance where employed 
in the ccnmunity but residing in school^ and this lack of correlation 
continued into phase five. This^ I think^ points out one very 
signigicant thing. We can have much more confidence in our test- 
ing procedures and our evaluative procedures when we are looking 
for a next step than idien we are looking several steps ahead. 

This reinforces the principle that you have to keep re-evaluating 
from one stage to the next. You cannot take the initial infor- 
mation and ejcpect it to remain equally valid all down the line. 

They did find^ in this particular setting^ that the field evaluation 
information was of more value than either the unit evaluation or 
the psychological test battery. They found that in the psycho- 
logical test battery the motor tests were the best predictors but 
here the criteria happened to be essentially motor tasks. It 
appears that motor tests predict eventual motor tasks perhaps 
better than other tests will predict other sorts of tasks. 

Another exanqple was a project Taylor at the Good- 
will Industries Workshops in Taccxna^ Washington^ with an intake 
battery of fourty-five tests ^ twenty -ei^t of which are standard 
measures^ commonly used^ and seventeen which they devised them- 
selves. The devised tests simulated the kinds of work operations 
which existed in the Goodwill Industries Workshop. There was a 
two -month rotation of jobs in the workshop and the clients were 
rated on each rotation by supervisors on seven scales constituting 
a work -evaluation scale. The question concerned to the relation- 
ship between the initial tests and the on-the-job evaluation made 
by supervisors. They found that less than half the tests were 
relevant to work performance, and most o^ these wer<» of dubious 
predictive value. That is, one could show statistically certain 
relationships; but in trying to apply them predictively to the 
individual case the probability of making a wrong guess is very 
high. Th^ did find a number of predictors idiich were fairly 
good. These included the digit -symbol test in the WAIS-Weschler 
Adult Intelligeice Scales, the Bender Gestalt Test, the Purdue Peg- 
board Assembly Test, and one of their own devising, a "wrap and 
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pack" test. These were the most predictive. But, I particularly 
want to note is that these were predictive for within -shop work, 
and no claim can be made as to their predictive effectiveness in 
outside enqployment . The results were not tested outside this 
particular shop, and idiat has been found to work in one setting 
mi^t not necessarily hold up in another setting. In effect, 
every shop has to test out procedures for itself to determine 
that they really work for them as they have worked for somebody 
else. Taqrlor found also that personality variables that were 
applied were non-relevant and did not give good predictive infor- 
mation of the kind that they wanted. Long run work efficiency was 
not measured. Again , this is a warning as to the limits to which 
one may legitimately generalize from a particular set of conditions. 

A third project was undertaken under the auspices of the 
Connecticut Association for Retarded Children by Warren Bower. 

This used on a fairly large sanqple of 572 adult subjects, mildly 
retarded, drawn from community and institutions, and both males 
and females. This is typical of the survey type of investigation 
where one goes out with a dragnet and finds a population that 
meets certain requirements. The population is then studied in terms 
of relationships of certain other variables. Bower set up four 
criterion varibles, that is, measures of adult success. Job success 
was defined as a nH rriimim of one year steady employment in a non- 
sheltered situation. A second criterion was that of a job level 
%diich distinguished those who were judged successful at holding a 
job at a fl-implft level from holding a job at a complex level as in- 
dicated by the Dictionary of Occupational . Titles . The thrid criterion 
measure. Bower called, w^ "social adjuel«ment," but it mi^t better 
be termed "practical intelligence," since it consisted of the Vine- 
land Social Maturity Score minus the Stanford -Binet Score. It 
seemed to me not particularly useful as a criterion of social ad- 
justment. His fourth criterion was that of community adjustment 
idiich was based on court records.. The well adjusted person here 
is sinply defined as one that Itias not gotten into trouble. This 
is a useful kind of information, but ratoer narrow as a definition 
of community adjustment. In general, the job success and the job 
level which really can be combined into one scale were the most 
meaningful of Bower's criteria. 

Bower employed twenty-one different measures from a num- 
ber of standard tests, the Stanford-31 net, the Vineland, Raven- 
Matrices, the California Achievement Test, Porteus-Maze, and in- 
cluded the Rorschach. One of the main reasons I am referring to 
this study is that it does utilize the Rorschach ^diich is not us- 
ually regarded with the mentally retarded. In his analysis Bower 
found that out of twenty-one measures, fifteen yielded wei^ted 
values for job success and this included some of the Rorschach 
measures. Four of his predictors had valid wei^ts for job level 
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and six each for social adjustment and * 

thing I am particularly concerned to point out that a , 

of test scores could yield a composite equation that would ^ ^ ^ 

predictive for job success, where no one 

L. But let me point out also, a serious limitation in studies 
like that of Bower. Predicuion has here a very special and liM 
meaning. It does not really mean predictive he only showed that 
there was a tendency for individuals with certain coir9)osity test 
scores to be found among the successful group. The comparison of 
test and other data behavior groups whose success failure has 
ready been established does not furnish a reliable guide for pre- 
selecting those who in the future will succeed or fail. 

The study by Stephens is a complex one but I want to 

speak only of one feature of it. Stephens ^ 

variables vhich she assembled by factor an^ysis, xnto 17 prim^ 
varieties of adjustment success in retarded adults. She discovered, 
for example, that it is one thing to talk about the 
of the enroloyee" from the standpoint of the en?>loyer ^d his j dg 
ments of the etig»loyee, and quite another thing to t^k atout 
"responsible work orientation" which may involve quite ^ffei^t 
characteristics and factors. It is the distinction b^ 

tween "having good steady work" as over against being judged hy tte 
el^loyer^T! -good ste^ worker," .-nd the two do not nece^^ly 
go together. A third success factor is autotony, that is ^ 
abiUty of the individual to be independent in ^s pneral 
transactions. That category may also be quite ^®*anguisteble in 
terms of personal characteristics from being a Sood "“rker m 
having good steady work. let, another syndrome or factor is task 
orientation" as Stephens called it. This is a jn 

success on the job, is combined with negative social attoibutes. 

It is the case of the person withdrawn into his work where ''orh 
becomes the only thing with which he is concern^. 

against having to cope with the world in general, Stephen r®®«®^®h 
points out the ia^iortance, you see, of defining our criteria an 
determining precisely what kind of success and failure we may 
talking about. 

Finally, let me refer to Evelyn Deno and her work in Mn- 
neanolis, for a purpose which I think is quite relevant here. This 
is a follow-up study on several hundred dropouts from the MLnneapo s 
Special Education Program to determine their ch^acteristics ^ 
what has happened to them. One thing that she found is that among 
Tese dropoSts is less than half of the subjects have been referred 
to vocational rehabilitation agencies. The majority o 
had been referred had already been closed or rejected ^om the 
services at the time of follow-up. Of those that had been closed, 
rehabilitated, seventy-nine percent were in satisfactory 
categories. Of those that had been rejected for service, fifty-six 

percent were making satisfactory adjustmer.tr 



Another aspect of Deno’s study had to do with the nature 
of predictive variables . She and her group established three 
orders of predictive variables vjhich they attempted to validate. 

The first order consists of conventional kinds of data -age, sex, 
verbal and non-verbal I. Q.'s, reports on following instructions 
and estimated measures of academic achievement. All these are 
the kinds of information that we commonly put into the record. 

The second order was based on these and other data in the case 
so that a construct was formed about each student. These con- 
structs were of a higher order than the foregoing data and in- 
cluded professional judgment as to native intellectual level, 
functional intellectual level, attitude, influence of sub-culture, 
influence of family, etc. These constructs are frank but defensible 
opinions of how the student could be described. It is a more gen- 
eral order than the detailed test information. Then the third 
order, which they refer to as super -constructs, are estimates of 
the student *s potential, i.e. potential for academic benefit, 
potential for social development, and eventual employability 
potential. This is an overall sort of judgment using the lower 
orders but going beyond them in terms of general subjective pre- 
diction. Deno found that the higher order, the hi^er the pre- 
dictive value. We can have a very low order of predictive con- 
fidence in test scores and these individual items of data taken 
singly summation. We can have more confidence in the more general 
judgments made with knowledge of the test scores, but also with 
contact and other kinds of interaction with the individual. We 
can have most confidence in those higher level predictions. What 
this does, I think, is to reaffirm that we do not have to wait 
for a nice recipe book of test procedures in order to make good 
judgments about clients. We can use a clinical judgment, if you 
like, which is more qualitative than quantitative, which co-or- 
dinates and puts together the objective information and general 
opinions estimate as to the probability of success. So it may 
give you some comfort, then, to know that you can trust your 
own judgment without going into all the labor and hazards of 
getting thousands of test scores. Not that these are without 
value, but alone they do not give us proof of future outcome. 

QUESTIONS AND ANSWERS 

DR. RABENSTEIN: Dr. Cobb has consented to allow you to ask some 

questions. I think he has provoked a number of questions, at 
least in my mind, and possibly in your mind also. I think in- 
stead of trying to run the film now, we will schedule it later 
and. we will just give this period, over to questions and answers. 

So if you have any questions, may we have them at this tame. 
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MRo BREGMAN3 Regarding the Pamicky study, what was the predict 
value between various phases three and four => how well did three 
predict the outcome of four? 

DRo COBBs They didn't explore this too much in detail c They got 
some indications that from phase to phase the predictive value was 
better than junqping over phases o Their main work was concerned 
with the diminishing usefulr.&ss from the initial phase o 
MR, BREGMANs Would you say that from two to three was the hipest 
iust in a general statement? 

DR, COBB? Prom one to two. They measured in each phase the be=» 

ginning and the end of the phase. In fact, they found that the 
best information was from the field judgments of phase one to the 
final judgments in phase two. This was about the best finding, 
QUESTION? Please comment on research that would be most fruitful, 

DRo COBB? I thirlc that the research that really needs to be done 
is to start with a very well defined, adequate size sampling of 
youngsters in their mid-teens at least, where they are still in 
school settings before they actually come into the vocational 
setting at all. They should be selected as individuals who are 
some kind of criterion probably going to require eventual as= 
sistance in making adult adjustments. There should also be a 
very good control group. Then, on the basis of Instruments and 
measures that have been used, initiate a study in w hi ch we can 
validate the applicability of measures for a later prediction. We 
should carry such a group through to the point where we can really 
make a determination of the historical predictive value of these 
measures. We need good cross -validational techniques to enable us 
to see that the things that we are talking about are really rel- 
evant to the outcomes that we are interested in. Most of the studies 
that have been reported have been in either a very limited sit- 
uation like a particular shop or a particular institution or they 
have been much too inclusive in scooping up a heterogeneous pop- 
ulation, I think the kind of study that is needed is one idiere 
you have a large enough sanqple carried over a long enough time 
with caref;il enough control over the variables so that validated 
prediction equations can be established, 

DRo E, MATCH? Back in the beginning you had touched on the self 
interest in the predictive period of vocational adjustment. Taking 
this into the field and some of the things which have happened 
in terms of the client involvement in evaluation, what level of 
H/R"S are you referring to? 

DRo COBB? The history of work in this area is one starting with 
those who are nearest normal and then rather slowly moving to those 
idio have greater severity and defects, 

DR, MATCH? Would you feel fifty is the cutoff point? 

DRo COBB, I ic not like cutoff points particularly when you are 
talking in the context of adult adjustment, loQ, measures are 
probably the least relevant; they are orJ.y broadly relevant. In 
gereral,, ycu can say that as you go down the I,Q, scale there 
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are certain changes in general expectancy. But nrtu^ly every 
study that has used the I.Q. as the variable in relation to adult 
adjurtment, has shovin it to have very little predioti^ve value. 

Coming back to the question of self-image, this has been a part- 
S area of interest of mine and there Is I think, very strong 
evidence that some aspects of self -referent tehayior are appropriate 
almost as far down as you want to go. What hM happened ^ the 
past is that so many retarded, particular:^ of the moderate to 
severly unpaired have been given very little opportui^ty to de- 
velop any^ense of identity or aspiration levels. Tte development 
of M adequate picture, or sense of self-esteem is extremely dJ- 
fioult under the conditions in which most of the retarded have ra- 

DR. MATCin Sir, are you describing a work adjustment progrm? 

Should we program and train for acceptable middle class social 

DR?*roBB: No, this of course is an extremely difficult thing to 

handle. But again we have to see the appropriateness 

these criteria in terms of social and cultural settings in idiich 

people are going to function. « 4.. a 

DR. MATCH: Going back to the second part of the question, what 

bout the client involved who is mentally retarded? 

DR. COBB: You mean by client involvement? 

DR. MATCH: In the decision making process. f^iriv 

DRo COBB: I think a degree of this is possible, even with y 
defective persons— in terms of things they like and don t like, 
things that they prefer not to do, the kinds of sett^gs they find 
congenial and non-congenial. The ability to make independent 
judgments, of course, is much more limited. I tMnk that we have 
toTo pretty far down to a point where determinations can be made 
fully from the God -like position of the counselor. We re still 

with people. As long as you have the person, you have some. 

body that is respondent. . ^ ^ 

DR. MATCH: One other thing. Is there a difference in the perfor- 
mance of males and females? ,, xi. 4. 

DRo COBB; Yes, there are. I would not want to spell them out. 

But I think the evidence does point quite thoroughly to impor an 
sex differences in behavior. 

AUDIENCE: Are these significant enough that we should be thinki g 

of doing different or separate things? , , , . 

DRo COBB: I think so. I think that this should always be a q 

tion in your minds, so that you don't wed yourself to a single 
pattern. I think that the evidence is going to show some 
tant differences here, but perhaps not in all the areas we might 

think that they would. . , _ , 

AUDIENCE: Are there any data or studies of M/R's who were luc j 

enough not to get into special education? ^ ^ 

DR COBB: Yes, a rather interesting follow-up study by Ballard 

and Charles in which they had a population from two sources - one 
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from those judged to be mentally deficient in public schools back 
in the early 30's, and then a control group not so judged but in 
some other respects comparable. These were followed up until f^rly 
recently; the final reports are now in, and one very striking find- 
ing from this was the upward change in intellectual status — I 
should say the change in rated intellectual status — over a period 
of years. This was taken by Charles to indicate a re^ constitu- 
tional change in intelligence. One can account for this in terms oi 
the social contexts from which these children mainly came, and the 
change in social contexts in which they later found themselves . 

Or one can interpret in as Ballard intended to do, as reflective 
innefficient measurements in the early stages. But tMs gets us 
into the situational factors which affect even the nature and meaning 

of test scores. 

AUDIENCE; Back to one of the statistics that you mentio^d in 
with the BVR's turning down I^/R's who were after successful, shoul 
we jump on the bandwagons with what they are doing with keeping 
retardates in school until twenty -two to twenty -five ye^s o age. 
m COBB; I would certainly combat the old notion now that oe- 
cause the individual has a lower Capability, therefore he doesn t 
need as long a schooling. There may be some things said, to the 
contrary that because he has a more limited resources, he may 
need longer formal schooling in order to make the most of it. 
would not want to make that as an arbitrary generalization because 
I think it depends a great deal on the kind of educational sett^g 
you have these younsgters in the same classroom or the same setting 

ATOIMCE; Tolstoi has advocated and come up with a work study, work- 
adjustment vocational program in educational * 

DR. COBB; Ri^t. Certainly I think that one thing that is 
cated is a much closer tie between the vocational process ^d. the 
educational process so that the Rehabilitation agency reaches down 
into the schools and picks up these people at younger ages . The 
schools have to reach out into the vocational function of the com- 
munity to co-ordinate their programs. . 

AUDIENCE; I would like to question whether the dissemination i 
research itself. Before I do this, without any comparison, I 
how many of those among us have heard of this rese^ch or read it? 
DR. COBB: Alright, how many have heard of these pieces of researc 

that I have been referring to? Raise your hands, a^body who has 
heard of or read all of the studies that I have mentioned. A few. 

AUDIENCE: Obviously most of us have not heard of this. Is tUs 

our fault? Or is it the fact that the dissemination of this in- 
formation is just not readily available to regions in general. I 
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can ten you one thing though, one of the studies is very similar 
to the program we provide. We have never heard of this study and 
certainly I would like to take it back and somehow use it in our 

our own particular purpose. How can we go about getting 
this information? & 

DR. COBB: Alri^t, I'll give you one source. One thing that we 
have been doing in my particular shop is putting together an 
^notated bibliography, "Predictive Assessment of the Adult Re- 
tarded for Social and Vocational Adjustment." While I know it is 
incomplete, we combed everything that we could possibly get hold 
of and arranged the materials into a number of different groups: 
descriptive studies before 19U5," "descriptive studies after 19ii5," 
personal variables," "studies on environmental variables 
"proactive studies," and "trends in training" - these have been 
put t^ether in mimeographed form and a number of copies were sent 
into VRA as part of our project report. But this gets really to 

reports, you see, are not them- 
selves widely distributed because they are not published throu^ 
the regular journal channels. ^ 

AUDipCE: Can an agency contact VRA? They are very good about 

sending out materials. 

M. cop: This is the general procedure. Except now VRA says 
that they cannot handle throu^ their own offices all these re- 
^^^ 4 . 4 * ^ several cartons of the bibliographies in my office 

saying that they want them. 
AUDIENCE: Who is the general agency there? 

DR, COBB: For this one, the Department of Psychology, University 
or South Dakota, Vermillion, S. Dakota. This is "The Predictive 
Asspsment of the Adult Retarded for Social and Vocational Adjust- 
ment. Part I, Annotated Bibliography." I will leave this copy here 
so that if anyone wants to refer to it, they can. 

A^EMCE! Then we may contact the originating agency for copies 
of project reports? ® ^ t' 

DR. cop: I am sure this is true for almost all of these studies, 
originating agency, they can probably supply them. 
AUDOTCE: Were most of these studies based on the mentally re- 
tarded population? 

DR. COBB: Yes, they are all on the retarded. 

Km EGERpN: Jfey I ask one question in regard to inforraatbn and 
dlssepnation? It is a serious problem. One of the purposes of 

exchange information; we cannot spend all our time 
With journals. So this is one way we hope as a Research & Train- 

ing Center to get some information desseminated. Thank you very 
much for coming. ^ ^ 
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Since early in the 1900' s, professionals have been evalua- 
ting and attempting to assess a variety of factors existing in persons 
and from this assessment we have tried, to predict later behavior . 

The present "work evaluation movement" began as early as - or as 
recently as 30 years ago. 

Historically man has always attempted, to assess and e- 
valuate other men and women. In primitive tribes, a variety of 
indicators were used to determine whether a person might be a war- 
rior, or a successful hunter, or a priest in the ten^le. Back in 
Bible days priests were dedicated, or these were other signs that 
people observed which they thou^t might indicate other career po- 
tential. In the early days, man believed that future occupations 
could be determined or were determined by exterior forces, such as 
the stars, or the phases of the moon, or the winds, or maybe even 
charms hung around ones neck. Later we believed, that man was not 
affected, by nature and. exterior forces and thou^t that they had 
no control over man's behavior. Now, we know at least, that the 
tenqper ature , sun, the barometric pressure, and weather will affect 
our behavior. Later, man began to observe that others were endowed 
with certain talents which if they could discover what they were, 
might point to a particular career. 

It was then gradually realized that man's destiny no long- 
er depended on exterior forces, but on these talents or endowments. 
There were early efforts to measure these differences. A sample 
of this early effort was the study of phrenology and measuring the 
bumps on one's head. This was one way they assessed, talent. At 
one time of our history, certain areas of the brain were believed 
to be better developed than others and certain talents were located 
in those areas . Charts existed of a man ' s head, where the artistic , 
musical, and mechanical talents were located. All one had to do, 
was study and measure the person's head, and find, where the bumps 
were. If you knew what was under that bump, you would know what 
talent he had. In Cleveland, about l5 years ago in the Yellow 
Pages, under the Vocational Guidance Agencies, wae listed an agency 
which used phrenology as one of their techniques. This caused 
great furor, of course, among vocational guidance people, so we 
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at least got them out of the Tellow Pages. This was an idea that 
was used at one time and was a part of early psychology and attempts 
in philosophy to help us understand each other. 

The science of physiogoiny was another where the shape 
of the body was an in^wrtant indicator. At Western Reserve University, 
we have scientists studying the man’s body frame. They say that 
they can predict whether a person is more apt to have a heart dif- 
ficulty, or might be more athletic than others, etc. They do some 
of this work at the Work Classification Center of the Cleveland 
Area Heart Society. They are using man’s appearance as part of 
their study in evaluating heart cases. 

You will remember that our forebears believed that long 
hands and tapered fingers were likely to indicate an artistic or 
musical ability. Ever today, we don’t take stubby fingered people 
who can’t reach an oct :ve and try to make piano players from them. 

Men began, in the early 1700’s, to become concerned with 
man’s thinking process, and that is nAien the beginning of our pre- 
diction of individual achievements began. People assess each other 
all of the time wothout recourse to any scientific methods, "girl- 
watching" falls into this catagory. Men have always sized up each 
other and probably always will without worrying very much about 
validity and raliability. But professionally speaking, this ob- 
servation has been of a scientific nature for about 200 years. 

In our special field, scientific observation has been used 
for perhaps 30 or UO years at the most. Professionally evaluation 
is fascinating and challenging because our subjects are very ^- 
teresting people. But we play by certain rules which are designed 
to make our observations more objective and useful. 

It is natural that work evaluation holds so much fascin- 
ation. What can really be more challenging or puzzling than a 
gifted individual or a client for whom we seek to discover in all 
the maze of his behavior, a key to unlock his closed personality? 

What can really place more challenge on our banks of ej^erience 
than the limited person for whom we are seeking an occupational 
hope and a way out of his unproductive existence? 

Walter S. Neff in the March 1966, issue of the Personnel 
and Gui^arice Journal has an article which describes the diiierent 
approaches which I^m going to discuss The approach that we use 
in Cleveland is one that comes under the discussion in his article. 

He lists several kinds of work evaluation and tells how they developed. 



IDr. Neff is a Professor of Psychology in the Graduate 
School, New York University 
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First of all, he talks about the "mental testing approach." 
This morning in our first lecture we heard that the mental testing 
approach is uc^d as a part of the evaluation of clients at Johns- 
town. Other approaches were, "job analysis approach," "the work 
sanqple approach," and finally the "situational assessment approach." 

Neff states that the mental testing approach found its 
be ginnin g in about 1900 in the work of Binet and Spearman. VIhat 
began as an effort to measure almost every conceivable kind of 
achievement or ability and even man’s ability to work. Mental 
testing received it*s strongest in5>etus in World War I. Psycho- 
logists were atten 5 )ted to screen individuals ^o were drafted and 
at least in early days of testing tried to select the readers 
from the non-readers, because th^ were finding that their usual 
methods of selection, weren’t working. They were getting large 
"dropouts" from their training programs. The army often selected 
people by lining them all up and then counting off.. The I's become 
cooks, and the 2* radio operators, and the 3's infantry men, and 
so on. Well, this was alright in the old days but at the time of 
World War 1, our first airplanes were beginning to be used, there 
were trucks, early tanks, and we had a little simple radio set and 
and a wireless telegraph, so it was becoming necessary to train 
people in these things. We were drafting men from all over the 
USA and many people did not have much education. 

This means something to me because my dad had an e^qper- 
ience with this kind of selection during the first World War. He 
grew up in a little town in central Ohio and dropped out of school 
scHoetime in grade school and became an employee of the local livery 
stable. The livery stable was a place idiere you got your harness, 
buggy, and wagons fixed. The steel was put back on the idieels and 
there was a blacksmith’s shop associated with this. If you have 
never seen one of these, you’ve seen it pictured in the Western 
movies. I'm sure. When automobiles first came into this community, 
it was natural that they take them to the livery stable, there wasn't 
any other place. You were licky to get gas for your car without 
worrying about whether anybody could fix it. So there were four 
automobiles that came into this little country community, probably 
a number of years after they were in other communities, and so dad 
learned to fix automobiles, at least up to a point. One day a farmer 
idio was a new owner of a car, was driving it into the bam and for- 
got how to stop it and he yelled "Whoa" to the car but it continued 
on through the bam until it landed in idiat you know is out behind 
most bams. So he proii 9 >tly gave up his driving career and sold the 
car to dad idio became one of the first young people in that com- 
munity to own an automobile. In typical fashion he was drafted 
and the Army put him in radio operators * training, not in what he 
knew something about, although perhaps they didn't need any mechanics 
S<Mie of you who were eating Army food as late as the Korean War 
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may wonder if they stiU aren't using the same kinds of techmques 
o/selection. After World War II, psychologists were still de- 
signing ways to select Army cooks. 

Now, the advantage to the mental testing ab roach iri 
assessing people is that it is quick and easy to administer, it is 
taespe^TO to use, and it is objective ^ reliable u^r ^timum 
conStions. However, we found that psychological ^sts, even 
though they have good reUability sometimes ^'t have P^c- 

tive value. Sometimes we don't know much either about toe conten 
of toe jobs that we are suggesting that some people perform. 

One of toe main problans ( there was a question earlier 
about the General Aptitude Test Battery and whether it wm used 
with everybody and what their e:q>eriences were) is that it is 
designed for specific populations who are able to produce in a yiick 
effective, organized, and efficient way. To do this, one has to 
be fairly secure and not be disturbed by inner or outer stimuli. 

One has to organize his thou^t effectively and put them down very 
quickly in a very specific way. Considering this, we can see w 
the psychometric approach is not one of the best ones nor is it 
appropriate for some of our work evalues who are immature or cul- 
turally deprived. Some have severe personality problems and the 
other problems which bar their accurate response. 



All too often, back in the past we had to use on^ tests, 
and vocational counseling interviews to assess people. Half of 
our case lc„d we had to write off because their profile on the test 
was low. We wished there were some better way to evaluate, but 
there wasn't any place to send people. Testing wasn't a satisfac- 
tory approach and it didn't meet all of the needs. We still use 
it today as a part of our total evaluation. 



During this same period, Neff states, another approach 
developed. This was called the "job analysis approach." This 
wasn't centered in the university but in industry. "Job anal^is 
was developed in industry, and as the title iRQ>lies it foOTsed 
on the job to be done. It didn't emphasize the characteristics 
of the work, just the content of the job. Neff says that th^ 
calls attention to the job or its tasks was one of the main vir- 
tues of this approach. One of the criticisms of vocational coun- 
selors has been, is that we don't know much about actual work or 
the actual nature of work. We may be recommending jobs to coun- 
selors as evaluators idiich really aren't realistic. We have an 
academic understanding of this, but we don't really know what the 
job is. We have little preparation for knowing whether aptitudes 
actually fit certain job demands. 



You remember the story "Cheaper by the Dozen" and 
Gilbreth idio was the engineer who had his family programmed even 
down to the last moment. 
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No thought is given in this approach to the individual 
performing the work. In fact^ the person is considered almost a 
part of the machine and maybe its weakest part. The efficiency of 
the worker was raised by cutting down all of his extraneous move- 
ments. They ignored the fact that a worker was really a soci^ 
being and enjoyed being with other people. This method contributes 
to the evaluation scene ^ however, by helping us understand what 
is involved in tasks which we may be recommending to many of our 
clients. This job analysis approach has added an in9>ortant con- 
tribution to the rehabilitation field, and that is that it has 
frequently stressed redesigning the nuichine or the job for the 
e^^)loyee. The problem with the job analysis approach for the 
handicapped, is that al l of the focus is on the job itself. Fre- 
quently, enough enqphasis is not put upon the individual and this 
philosophy ha s not advanced too far in the rehabilitation field. 

The next technique he mentions is the "work sanqple." 

The "w' *k saiqple approach" to evaluation is one in idiich our agency 
in Cleveland ha.s been particularly active. Neff points out that 
this is a post World War II phenomenon. This was an effort to 
put together the virtues of the psychometric test and the job 
analysis approach. From the psychometric testing approach, the 
work evaluator took over the ideas of standardization and statis- 
tical rigor. Prom the job analysis approach, the work san;)les were 
anri effort to evaluate the client ly observing him actually at 
work. Early apprentices were tested in this same way. One 
began an apprenticeship and if he did well for a period of time he 
could continue. 

What is a work san^le? It is a sample of work. On the 
^RS evaluation floor we have set up the operation of a rocker - 
arm assembly, idiich would normally fit on the top of a Ford engine. 
The job is set up in our small work evaluation department much as 
it would be in the Ford engine plant where they make about three 
to four thousand engines a day. The client is asked to assemble 
k 7 rocker -arm assemblies. Neff says that the work sample is a 
mock-up or a close simulation of the exact industrial operation, 
being very similar to the kind of work the en5>loyee mi^t be ex- 
pected to perform on the job. Neff states that the developer of 
a set of work samples must start, therefore, with a detailed job 
analysis of the kind of Industrial operation that he wished to 
predict. Then he must take the job from the shop or the conq[>any 
into the laboratory, idiich is the work evaluation department. Then 
after standardizing it and developing the suitable criteria for 
speed and quality, assembling the tools necessary, then derive a 
method of making observation of the client's work behavior in pro- 
duction. Ideally, then, the work evaluation department must run 
a number of people throu^ this; then get some kind of standards 
and norms. In some work evaluation departments, this is done. 
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But the main difficulty comes in doing a follow-up study to deter- 
mine what the accuracy of the prediction has been^ or to bring 
about other statistical treatments of what has been done to see 
whether it is really valid. Frequently, standarization and val- 
idation is neglected and work evaluators assume that their pre- 
diction has been correct. This is irtiere the work sanq)le approach 
comes under the most severe criticism. All of us run into this 
criticism. Particularly from researchers and our own research 
departments . 

In the early days of the work san;)le approach, and in 
some areas today, much energy is ^>ent in trying to develop what 
we have called in the past "actual work saiqples" or the exact 
replica of the job and the work process from industries within 
the specific communities. I heard our first speaker this morning 
talk about how iiqpossible it would be to bring in actual work sam- 
ples from all the communities from which these clients come. Of 
course, it would be! Of course, one can never exactly duplicate 
the Job. If in Johxfstown, you went down into your industrial area 
and you picked out exact jobs and put them in this setting, it 
still isn’t the exact job. Again we can use on our'aelves the same 
criticism that was used on the job analysis approach.' Workers are 
a part of a working social family and the social experience, the 
heat, the noise, the motivation, and wages vary so considerably 
in our shops that there is little con^arison between them and in* 
dustry. We are really not measuring the actual job but it ttxsk 
us a long time to come to that realization. 

We have ended up with a series of work like tasks. 

They are work sanqples, work tasks, or job sanqples. They are 
useful as long as we don't delude ourselves into thinking that 
they are actual work. If a number of assembly jobs exist in 
your community that might be suitable for your client, then you 
would perhaps want to go out and get a number of assembly tasks, 
but not, of coarse, duplicating every assembly job in the community. 
If a number of clerical positions are available in your community, 
then you would get into a number of clerical operations. 

Another problem with trying to reproduce the actual job, 
we have learned, is that jobs, of course, are changing very rap- 
idly. The climate of jobs in idiich we all live is changing very 
rapidly and we could never really keep up. The point I am trying 
to make is to get away from thinking about the actual job sanqple 
and think just about work sample. 

Neff has asid that he questions whether in our work 
sample approach, if we are predicting actual job performance or 
predicting the client's ability to learn something' in a training 
situation? Our answer to this is that we are not predicting actual 
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job performance but predict whether we think he is en5>loya.ble, 
whether he is trainable, whether he shows he shows he is sheltered 
or diver sional shop material or not en5>loyhble at all* We think 
we can do this • We do predict his ability to learn something in 
answer to Neff's question - in a training course, in work adjust- 
ment, or in other skilled training, or classes in our community. 

We observe, for exanqple, in doing work samples how he follows 
directions, how rapidly he learns, his concentration, his accuracy, 
craftsmanship, and so on. 

The "TOWER system," as many of you know, was developed 
by the Institute of the Crippled and Disabled in New York City 
and was certainly one of the earliest and the finest efforts in 
the job sample technique. These attenpted to replicate the exact 
work in the New York community and they tried, for this, to pre- 
dict trainability and employability in these jobs in the community. 
The "TOWER system," and some of the criticism that we have had 
for the "general work sample approach" have also been a]^licable 
to the "TOWER system" as we bring it away from its setting and 
try to put into our own community. The TOWER people ri^tly ad- 
vise all to standardize these jobs on local populations and c<mi- 
raunities. Some of us have not learned to do this and have used 
the "TOWER system" to later find that it doesn't predict well* 

We use certain aspects of the "TOWER system" just as you do here 
at Johnstown. 

In our experience, the strongest support for the work 
sanple method is that it's about as close to the reality of work 
as we can get within the rehabilitation center, except for actually 
putting clients to work, in the center. But if we are in a hos- 
pital or the laboratory or the rehabilitation center, the work 
sample is just about as close as we are going to get. It gives 
the evaluator an unexcelled opportunity to observe production, be- 
havior and work attitude, sometimes for a long period of time. 

It was with interest this morning that I listened to 
the plan to collapse the evaluation into a short two or three 
day effort. We tried this by pressing, working and structuring 
evaluators. We were able to get our work evaluation down to nine 
days. We started out originally with an ei^t week's evaluation 
and then this time diminished to six weeks, four weeks, two weeks, 
and eventually down to nine days. Since that time, we have become 
unsatisfied with the short evaluation and have expanded again. 

Now we have a much longer period of time for client work evaluation. 
Clients need a week to get settled in and to relax to show that 
they really can produce. No attenpt is made to evaluate until 
they can be part of the scenery and part of the group. This brings 
out the point that the job sample approach allows the client to 
dissapate a lot of his anxiety and concern so that he can perform 
his work more suitably and give a better account of himself . One 
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of the prime faults of the mental testing approach was that it was 
a "one-shot” assessment. With the Job sample approach , you can 
give Job sanqales a number of times to watch the change in the per- 
son's behavior which is one of the significant things for which we 
are watching. The work san 5 )le approach allows for several trials 
under controlled conditions where the client can be observed of 
settings. The client is in the department for extended periods 
of time and this allows him to develop a rapport or relationship 
with the evaluators so he can produce with help and support some- 
what near the level of his actual potential. We can observe, too, 
his speed, rate or learning, accuracy, craftsmanship, and relation- 
ships, and so on. 

Neff criticizes the reliability and validity of the work 
san^le approach and cites one of the major problems as the obsol- 
ence of the work task. We know they become obsolete and that they 
are verv difficult to validate. The criticism is a Just one, but 
this doesn't mean that we throw them out. We have done a recent 
study of 207 cases where our evaluators predicted, on the basis of 
the "total experience" not Just the individual work sample, the 
outcome of this person in the eu 5 )loyment market . Interestingly , 
they were accurate in predicting eii 5 >loyability 7h% of the time. 

We use the total experience, the conference, etc to predict whether 
he is enqployable , whether he has sheltered en^loyment potential, 
and Aether work adjustment training will be successful.^ That 
is one of our first efforts to study the global process in devel- 
oping what we are doing. The validity score, is satisfactory 
when comparing it with some of our psychological tests. 

The fourth and last approach is the "situational 
approach," as Neff calls it. It originated not much earlier than 
the middle 1950*s. It is simialr to the work sanqple approach in 
that it attenqpts to duplicate actual work. I think my first con- 
tact with this was with Dr. William Gellman^ of the Jewish Voca- 
tional Service in Chicago. We used to have much discussion about 
this back in the days when we had our VRA grant to develop a work 
sanq[)le approach. He had as part of his work adjustment prototype 
project a two week evaluation process. He evaluated clients in 
the work situation at the contract work bench. His position was 
that we didn't need work sanq;)les. We were committed for five years 
to develop the work sample approach by the VRA. We knew that 
"actual work" was a good way to evaluate people but this devel- 
opment came later for us. 

Neff alleges that the situational approach asks a prior 
set of questions, and I am saying that it does, but so does the 



^ Dr. Willaim Gellman is Executive Director of the Jewish 
Vocational Service. 



Campbell 



work sample approach* He says that the situational approach asks^ 
"Can the client work at all?" "Can he conform to customary work 
rules?" "Will he take supervision?" "Can he get along with other 
people?" "Is he flexible?" "Can he put In eight hours a day?" 

"Doe*> he produce rapidly?" and so on. His contention Is that the 
work sairqple approach emphasizes skill potential and. the situational 
approach focuses on general work personality* I think this Is an 
artificial distinction* Both focus on these* It may be true in 
his e^qperience, but certainly it is not the situation as I see it* 

The work san5)le approach also asks the same questions* 

Our work evaluation department was developed ten or 
twelve years ago, but now this in mid -I960 's VGIS work evaluation 
is a combination of the work sanq>le and the situational approaches* 
We have tried, to put together the technique that have been found 
most useful from both approaches* Psychological tests have been 
added as well as medical evaluation* 

The work sair^le abroach itself is slanted in the di- 
rection of a general evaluation, as I see it now* The situational 
approach focuses more on the clients performance and study of him 
as a worker doing actual contract work* When we put these two 
things together, some of the problems of the work sample approach 
are eliminated* The work that the client does is real and there 
are deadlines and quality and production standards which are all 
real* He may see the work that he has done this morning go out 
on a truck this afternoon and realize for the first time that he 
has produced something worthwhile* On the other hand, all clients 
are not really ready to go into production situations and need 
much individual attention* 

We use the work samples with a number of our clients 
before putting them on real production* Real production belongs 
to somebody and has real value* If you have a retarded young per- 
son and don't know whether he can cound twenty-five pieces in a 
bag, you better find out before you let him to do it for an hour * 
This is where the work sairqple comes in handy, in learning a great 
deal about the client and getting him settled into the situation 
before you turn him loose on real work* The situation always has 
to be an individualized* Another reason we added the situational 
to our work sample situation was that occassionally people did 
not do well on the work sample approach as in psychological testing* 
We were about to write them off and so fortuitouc-^'^ ^ ound he pro- 
duced better on real work than we could have predicted* We 
learned that we should try real work with everybody* In some sit- 
uations start him on real work first because the work sample ap- 
proach can be threatening* In our early work evaluation depart- 
ment we maintained an air-conditioned, quiet, nicely painted, cor- 
rectly arranged clinical atmosphere* It was not near the rest of 
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the agency and the only time the client approached a real worker 
was if he made a mistake and turned in the wrong direction, or we 
happened to take him through the shop because we thought we might 
want to go to work adjustment. Later the work evaluation depart- 
ment moved near the end of the shop floor where the client cou^ 
see the work ahead was noisy, dirty, and pressured. We did this 
wondering about the drop-out rate and found that it actually de- 
creased. In fact, the drop-out rate among work evaluators has 
even in 5 )roved because they were less isolated. 

Neff indicated that one criticism of the situational 
approach is that usually the rehabilitator has a small variety 
of work which the client can do. For this reason, he can®t dup- 
licate in his little workshop all the other jobs that exist in 
the community. This is the same problem we have with the work 
sample approach. He can’t duplicate the levels of skill which 
exist in the community. The professional evaluator’s answer to 
this is that any kind of work is satisfactory so long as it gives 
the evaluator an opportunity to assess the client by observation. 
This is a weak argunent, of course, but it still is an argument. 

I think the stronger point is the fact that most clients who are 
referred for evaluation service in a community vocationally oriented 
rehabilitation center, in our e^qperiences, are virtually unen 5 >loy- 
able to begin with and come from a limited educational, cultural, 
and experiential background. Ifost of these clients by necessity 
will need to begin at the low level beginning classification of 
jobs. Very few of our people in the community centers have the 
skills and the potential that your clients have here. The work 
evaluation and the situational setting matches closely the duties 
and the tasks which will be required of him in his beginning job. 

In this sense it replicates the work to be done in a fairly ac- 
curate way. If you think back to your early first job, most of 
you, will realize that it required, very little skill. One of my 
first jobs was sorting rotten potatoes from the good potatoes in 
the back room of an A & P store, and this didn’t require any 
fancy evaluation or a great variety of tasks to know whether I^ 
could do this in the 93 degree heat. Most of us begin simply in 
the job market, unless we have advanced job training and education. 

This final project report, now out of print, outlined 
how a person could begin a work evaluation department in his own 
agency. The report outlined the floor plan of the work evaluation 
department, the means by which job assignments were made and direc- 
tions for timing and scoring task selection sheets, work evalua- 
tion interview forms, report forms, and so on. Guidance in writing 
reports and. conducting conferences of the work evaluation type 
were also included. 

The preliminary report concerned the "how -to -do -it -cook- 
book" material and the final report contained it in some validation 
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studies \fhlch were disappointing. We did take a number of our job 
samples and compared them with performance on tasks and. analyze 
them in the same way that the U.S. Enployment Service studied 
workers traits in some of their UjOOO jobs. We tried to show how 
some of our tasks related to those traits so that if we could, 
discover these traits in our people, we might be able to match 
them with the traits listed in the Worker -Traits Manual. We also 
followed up on 192 cases randomly selected, out of 1,000 cases that 
we have studied in five years and followed them up to see what had. 
happened to them to try to establish validity. We have found at 
this stage of development, that we must use the total experience 
to predict what is coming for the client. 

Our young field, of work evaluation faces the same pro- 
blem that vocational counseling and psychotherapy faced some years 
ago. We are trying to move from an art to a reasonable facsimile 
of a science and. I am not sure we are going to make it. It may 
always be an art; we don't know. Clients are being appraised, in 
work evaluation departments throu^ the country by rule-of-thumb 
methods; the value of which often depends upon the clinical com- 
petance and. sometimes the scientific competance of the individ.ual 
work evaluator. There is not a full-fledged standardized, training 
program, at this point coirparable to thax. offered clinically and 
psychologically by counselors, or vocational rehabilitation coun- 
selors in which accepted skills and competencies in training indi- 
viduals are transmitted to students. In a short time the first 
graduate school for work evaluators will start at Stout State 
University in Menominie, Wisconsin. 

Now we can only guess what kinds of individuals using 
what methods based on what theories are carrying on work evalua- 
tion. In such an unstructured, situation we can expect performances 
in our work evaluation departments ranging from inspired, intuitive 
judgment, through intellectually barren mechanistic rituals. As 
with other professions concerned with appraising human ability, 
increased professionalism will emerge and that is one of the reasons 
for our meeting here. We have the rudimentary application of 
scientific principles in the field and we have begun to organize 
a body of technical knowledge to which the evaluator may turn for 
orientation and assistance. 

Someone asked here this morning about who should do work 
evaluation or who is doing it. Auburn University plans to train 
work evaluators in a number of shops around the country without 
benefit of degree. They found people doing evaluations who had. 
not finished, high school. A number of evaluators had a few courses 
in college and. other had. their college diplomas. 
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QUESTIONS AND ANSWEBS 

QUESTION: The Peter Pauhle study: Did he have a standardization 

sample? 

ANSWER: He used the same approach. He had an industrial engineer 

study each motion and fragment of motion that was involved in the 
assembly task that he had. They totaled all these times to get 
what the industrial standard was for that particular job, even 
though that job might not exist in industry. An industrial en- 
gineer can tell us what the normal worker time requirement would 
be. What he did was to get the standard and compare his patient 
or client with that standard to see whether he made it and at 
what level of efficiency was he working. 

QUESTION: But as I visualize our society today and probably more 

so in the future, it is not so much the physical movement that is 
going to be important. It is going to be more the application of 
education to work. I do not see that this is going to be too much 
of a benefit to study physical movement when education is a work 
factor. 

ANSWER: Well, if you are saying by that that you think that the 

personality factors or the attitudinal factors or that the social 
factors of work are more inqportant than the hardware, I would agree. 
QUESTION: Personality and education both? I don't think we can 

measure this by wiring a person up and .... 

ANSWER: No, he didn’t attempt to. He attempted to measure per- 

sonality factors in other parts of his experiment where he was 
testing the frustration and reaction, the heartbeat, the psycho- 
galvanic skin response, and a lot of other things in relation to 
how he responded under stress and how quickly he would adjust to 
those things. For me, most of what we get in our evaluation, the 
more important part of it, is what we label "work attitude." We 
have two main categories: work performance (which we describe 

his performance in assembling, sorting, and clerical kinds of 
tasks) and behavior and work attitudes. These are two main parts 
to the report. We do not repeat all the medical and psychological 
information in the report. 

QUESTION: • I think that repetition is good if you put it into 
functional terms. I don't think you need to repeat everything 
from the medical report. 

ANSWER: We would certainly have to consider the medical, psycho- 

logical, school, and work reports, but the report doesn't need to 
include them. Another major problem is the work evaluator's ability 
to mobilize to write. It is a difficiat thing to get people to 
commit themselves on paper. Evaluators will do it in conference; 
they will talk very freely about what they think about a plan and 
what they guess will turn out. But when they go back to their 
office with the dictaphone, it is awfully hard to get it out. 
QUESTION: Did Bauble's work try to find out what people with very 

severe limitations in their hands could do by analyzing the job 
and what was required. 
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A-'o/Ji-vl: I caii’t anGi'JGj.' your quootion. I knoij that he vian 

jetting; in v.’hich t'lere VJas nsverely physically disabled people. 
'4UE3TL.’'.: '/Jliat “was the conclusi'^n regarding the eriectiveness Ox* 

this method Tor predicting Tuture success? 

AI3US1: Peter Pauhle, like all researchers, said that more needed 

to be done, he said that a whole new series ox experiments could 
be carried out and that some oP us should pick up on these. 

4lIb7?I j;' : bid he consider his experiment successful? 

Yes. As a demonstration of research for another way to 
people, -ia was critical of our primitive methods. This 
scientific, exact and objective way to evaluate its believed. 
: Don*t you t}iink we rely too rauch on industrial samp?.es? 

V.h/ nob service area samples? 

ANSWER: IJe hvove evaluees assigned to the service department, main- 

tenance d.epai*tnent, and by that we mean cleaning, wa:cing and the 
like. We have them on the shipping dock. Some people may need 
this experience and we want to see how they can handle themselves. 
Sure, you can do this and you should make use of every facility 
and capacity you have in your center to evaluate people. 
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VJhen Kcrl Eccman first called me to tall: to you atout our shop, and 
iicularly after he told me the topic of the seminar, ’ Some Recent Ad'/ances 
^(-''^arch in Vocational E’/aluation,“ I thought to myself h^ really canno. 
serious. But, I guess, it can be true, because this operation vmich I am 
it to dcsciibe, I suoposc, had as little advance res^'arch as any prorjeco 
I 'now of. He have not attempted any research since we are really not 
' how -:iuch of an advance it is. Primarily, this facility is what we cen- 
to be an efficient contract .hop, an efficiently operated busir.esc 
ration where e-/nluntion is limited to very narrow, specific goals. I 
nk the one thing it probably does do fairly well is to compromise t;i»^- 
--ic shor conflict between production in a contract shop and the cvaiua- 
rproccss that is going on there. Those of you who may have had an;y'thing 
do v;ith sheltered workshops \rhere industrial subcontracts arc carriec^on^ 
w of this conflict. Either vour demands for production are so great that 
sannot do an adequate Job of evaluation or, if you concentrate on e^/alu- 
on, your contract commitments are not met. I think we have done this 

.rl: ’..’ell. 

The shop actually came about back in June of 1064 . The State of 
:.ont at that time was operating a demonstration project in the city o 
Dos^i. a small milltown, with a very large woolen mill operation. There 
developed traditional patterns of low income groups in the to\m, Pa^-erns 
.reifare^ dependency, and when the mills suddenly shut down in 19.-, it -cft 
town devastated. A veiy high number of vrclfare cases developed out of is 
uation. The Rehabilitation Division had a very tough time getting tnese 
nlc hack to work so a selected demonstration project was started to deal with 
situation. Now one of the problems of working with these people was onat ao 
point in the rehabilitation process the counselor liad vei-y Limited Knowl- 
c, or no kjiowxedge at all, of the potentials of the people he was working 
h He knew that they had developed some veiy undesirable dependency pat- 
ns; they, bv and large, had low motivate* on or no motivation at all: ana so 
as the counselor was concerned, they were complete unknovms for placement, 
project director mentioned to me one day, "if we only had a shop where we 
iLu expose these people to a work situation, we would know at least this much 
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about them: can they operate in a paid work setting and how do 

they operate? How do they interact with people in the shop”? 

They had at tae end of their second project year some extra funds 
left over, enough to rent a building and that is about alio 
Technically, this is how the industrial workshop began <> 

Going back a little bit, at that particular moment of 
time, I was connected with the industrial homework office in Vermont 
idiich is a work program for the severely disabled, who are home- 
bound either because of their disability or their geographical iso- 
lation or both. Over a period of ten years we had developed a 
rather workable program. We had a variety of sub -contract opera- 
tions. We had some temporary personnel that we could detach from 
our program and loan to the shop. We had transportation facilities 
we had a few trucks; we had an established payroll system and had 
a certain amount of knowhow. So with this in mind, we simply 
took what money they had for the purpose, we rented a building, we 
bought some work tables, and we took over a couple of industrial 
sub -contracts that were not particularly good for homework be- 
cause they really required that you set them up on a production 
line basis. This cannot be done in a homework setting because each 
person must work in isolation. We brou^t these jobs into the shop 
and brou^t in four clients from the demonstration project and we 
were in business. So you can see what kind of planning went into 
this thinjg. The purpose of the shop at that time was sinqply to 
expose the client to an actual paid work experience and drew some 
very rudimentary conclusions from this, and at the same time hope- 
fully build motivation for work. Ihis was done at the san» time 
as the welfare payments were being phased out. We hoped that we 
could interest them in the job that they were doing so the welfare 
payments could be dropped altogether. Our very first job was a 
machine stitching operation where we stitched little pieces of 
pre-cut cloth onto a cardboard header to be used as a pocket hand- 
kerchief. You have probably seen dry-cleaners use these as adver- 
tising gimmicks. We had four people at sewing machines, one doing 
folding, the other doing the machine stitching, and the other pac k- 
ingo This is where we started. All the work at that time was on 
piece rates. People earned exactly what they produced and our 
wage scale was very low; the shop average was about $15 a week. 

Some people earned as low as $10 some earning as high as $liO per 
week. 



In January of 1965, we had what amounted to be a cat- 
astrophic change. We had been working with the International 
Business Machine Corporation for several years in our homework 
program doing some very low level salvage jobs. I approached them 
and asked them if they could not use our shop which was only a 
few miles from their manufacturing plant. They sent a survey team 
down, and in January of 1965, th^ gave us a job of salvaging 
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transistors. This employed about ZC-2i> workers. I want yz-: tr, 
see what kind of work these people have formed.^ 

This is the transistor that they worked on„ 1 :g pro- 
bably cannot see it because it is so small. I can show you 
blown-up versions of the same component. This is the 
I*m talking about. You can see the relative size of It P-rme ( f 
these transistors have been manufactured by others nsve tee;, 
purchased by them and they were mounted on a cfrcuit boTdc Tb't 
entire circuit board had either failed during their inspection 
process or else it had actually been installed ir_ a data prefer *, v 
machine and later had failed. Rather than try to find the indi ^’ rj 
con^onent that was damaged^ they pull the idiole circuit oat 

and strip the con^sonents from them. And there is the little tran- 
sistor I just showed you. Now when those are put on the 
those three little leads have to be clipped over backwards sc 
they can be soldered on the back. VRien we got them, by the 
incidentally, they were bent, after being pulled out of the cir- 
cuit board and we had to straighten the leads and bring them up 
to a vertical position and then inspect for about fourteen differ- 
ent defects. This one here, for instance, has the edge aiouri 
the can bent. This one with the tab on the side has a very sirrll 
scratch on it. Incidentally, these all have to be worked on 
under an illuminated magnifier, obviously because they are so 'rety 
small. This one has the lead bent as I described to you befere. 

The con^)onent has to be thrown away if the lead is bent more thr:.. 
ten degrees in reference to the base. If less than ten degrees c. 
it can be straightened with a pair of small pliers, a»id Vrevghi 
back Into a vertical position. Some of these componeni? h^^ve 
scratches on the side of the can. These are hardly vislb'r *.z thv 
human eye without magnification. The insulator which made of 
glass and located on the bottom has to be inspected for o* 

for cracks around the insulator itself. The leads niay 

have scratches or dents on them that have to be inspected. The 
workers also look for scratches on the side of the can. On the 
tqp of this is a serial number ~ a three digit serial number. 

These transistors, after they have been worked on, hav° to be 
according to this serial or part number. The part number :arnot. 
be obic'jiM'. If you cannot read it then the cenponent has to be 
thrown away. There are probably fifty different series of there 
things and all that have to be separated and returned by part num- 
ber. Here is another picture of the serial number on the t-.p 
this thing. It is almost impossible to see without considerable 
magnification. 



Ijfr. Hill demonstrated the transistor circuttboard tc 
be explained. 
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This is the kind of job we got. This job allowed us for 
the very first time to establish a minimum wage in the shp of a 
$1.25 an hour and we have continued to do this ever since. The pay 
rate in the shop is based on productivity but is converted to an 
hourly wage and it goes no lower than a $1.2$ an hour. After 
February 1 it will go to a $l.liO an hour. Traditionally it has 
gone to a maximum of $2 an hour and probably will go to about $2.20- 
$2.30 an hour in the future. What we are doing you see, is bringing 
people in on a job which they may or may not be suited for, it is 
just simply to e^qpose them to this work situation and move them 
around the shop doing the various operations we have to offer them. 

The next job that we got was one that involved an advance 
in solid state circuitry with IBM. Essentially it involved con- 
densing this old circuit board to this miniture size. It is a 
little ceramic wafer, which has the transistors and resistors you 
see here - all printed on the ceramic wafer. These we had to hand 
load and then inspect visually for leads that were missing on the 
bottom of the ceramic chip and for the little solder tracings that 
might have broken or had not been correctly applied. This partic- 
ular job required an additional 25 or 30 people. This raised total 
shop eii5)loyment to some 50 workers. 

In January of 1966 IBM came around again with another 
inspection operation which meant we had to hire an additional ei^t 
people. To make better use of the very expensive equipment which 
they loaned us, we had to go into a two -shift operation and use 
this equipment I6 hours a day instead of eight hours a day. So 
we now had a shift coming in to work nights. This brings us up 
almost to the present time where we have approximately 70-85 people 
working in this shop in any given payroll period. This latest job, 
incidentally, was, I think, one that Dr. Egerman was particularly 
interested in because the component was so small you could hardly 
see it at all. It was a little chip, actually an entire printed 
circuit, about the size of the end of a sharpened lead pencil. 

These were all glued, in the manufacturing process, to a disc 
about two inches in diameter. There were about 1,500 chips on each 
disc. In order to inspect them they had to be put under a hundred 
power binocular microscope so they could be picked off and inspected 
with ease. About lu different inspection operations were performed 
on the component. 

The complexity of the shop and the volume of business we 
had gotten into necessitated that we divorce the shop from our in- 
dustrial homework office and set up its own administrative structure. 
The unique feature of the shop from a business stanc^oint is that 
our legislative emergency board, in its generosity and wisdom, told 
us that we could keep the shop going after initial grant funds were 
e^diausted only if we could make it self-supporting. They would 
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approve staff positions, but only if we could pay for these and nl] o)ieratine 
costs from our contract incoinc. To date we have been able to do it, lut you 
can see this imposes sor.c severe lir.dtations on vhat v/« are able to off^r in 
the way of evaluation services. This led to our decision to lii'it >'jr evalu- 
ation services to on-ohe-.iob evaluation only. 

Now let me read to you the factors we take into consideration in the 
evaluation process and by the nature of uhese thinf/s, it involve; '.h»- counselor 
at any Given period of time. First of all, v.v asl our shop personnel to ev'al- 
uatc clients on their punctuality" on thr ,iob and their abuse of break privi- 
leccs. Do they follow instructions? Do they accept corrections from shoo 
floor people? Do they concentrate on the .job on u"hich they ar^* placed at a 
.Given time, and if the jobs performed provide a fair opiX5rtunity for observa- 
tion, comment on the following things. Yo\: see, it may or m.ay not be an opera- 
tion that w"ould yield any useful information on the following factor.s: Dr. they 

demonstrate time and motion econony? Do they have the ability to organize 
their work, if organization is called for in this type of vrark? Do they have 
foresight in anticipating problems that they are going to have? Do they show 
initiative on the job? Can they handle their tools? Can they plan their work 
in such a v;ay that the v/ork flows along smoothly*"? And, arc they dependent on 
others in the shop, either shop personnel or co-v;orkers who are working next 
to them? V/e asked shop personnel to make an evaluation of their production 
rate. Production rate here, the norm, "t/ould be v;bat IBM itself has established 
as a production norm on this given job. Oddly enough, we have found that about 
50 per cent of the clients in the shop, brought in v/ithout any knowledge or skill, 
and put on jobs that may or may*" not be ^ good job for them, have Ghov.vi that they 
can produce anyv/here from 50 to 100 per cent over the IBM norm established for 
this" job. This would make an interesting research project for someone, but at 
present wc cannot account for this. Me asked shop personnel to evaluate them 
on the quality of the work they*" do and again, th_s is actually part of I3«I's 
production process, so you can imagine the quality control is very, very 
stringent. They send quali"ty control teams down at least twice a week to go 
over every operation in the shop to see how wc are doing it and every single 
lot of work that is returned is subject to their very stringent quality control 
standards. 



VJe asked shop personnel to evaluate them on their attitude toward the staff, 
on their relationship to fellow v^orkers. And this is a real problemi. Evaluation is 
just simply how they get along v/ith them, how they relate with them, can they work with 
a certain group of people v/ithout getting in trouble. Then they manipulate the workers 
a little bit; move them into another group over a period of time to see hov; this works 
out. How is the client making use of this opportunity in the shop? Is it some- 
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thing he is doing sinply because his counselor has asked him to do 
it? After he gets there, after he gets used to the pattern, is he 
beginning to toJce an interest in it and trying to do a good job? 
Should the client continue? If so, how long should he continue? 

What limiting factors have become evident while he is there, either 
from the disability or from other causes? If the shop floor people 
are asked, they make a judgement concerning a client on a given 
job and whether his disability is affecting it or any other cause 
that can be affecting it. What observations from actual shop opera- 
tions (and we underline actual shop operations), will be helpful 
to the counselor and client in selecting a goal for training or 
placement? You see we structured this question in such a way that 
we are trying to avoid personnel making any predictions about what 
this client can do. With such a rudimentary evaluation process 
it would be inappropriate for them to do any predicting. Wo what 
we are asking them to do is to observe them on an actual shop job 
and then on the basis of this, what information would be helpful 
to the counselor and the client in selecting a goal for training 
or placement. Then, if the client's stay at the shop is terminated, 
give the reasons and the present situation and then additional 
remarks . 



One of the things we have been able to do at this shop^ 
by conducting a well run business and l?y careful, detailed and hi^ 
quality work is to develop an excellent experience rating working 
with the International Business Machine Corporation. As IBM phases 
these operations out, because of the changing state of the art in 
electronic circuitry, and get into more coitqplex jobs, the Industrial 
Workshop is always in a position to accept the newer, more complex 
operations that they are developing. IBM knows that in the past 
we have been doing a good job for them. This also has enabled us 
to ask and obtain better prices on our product. This has led to 
constant high-level maintenance of our contracts with them and has 
allowed us to employ almost as many people at any time as the coun- 
selors want to send to us. As a matter of fact, wc have so much 
work at the present time that we cannot take care of it and we are 
establishing satellite operations of this shop in other sections 
of the state. We started one at the Waterbury State Hospital where 
mental patients are going through our rehabilitation program there. 
They are being exposed to this kind of on the job experience. The 
number of given people at any one time runs from 75 to 90. These 
people are earning about $15,000 a month. Our markups are more 
than sufficient to run the entire shop operation. Our total gross 
business is running about a quarter of a million dollars a year. 
This has been in just about a two-year period of time. 

Now we have two directions that we can go in right now. 

We can either extend this same concept to satellite operations in 
other parts of the state or we can apply the earnings of the shop 
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to develop what ws have in r?ora dep'ih . Vie have not quite decided 
what to do. Cur immediate inpression is that we will try this con- 
cept in other areas of the state and this way not have to get into 
the very hi^ costs cf professional personnel in a sophisticated 
evaluation program. The staff are non-trained, the shop manager 
himself has business experience, absolutely no experience in re- 
habilitation and none whatsoever in evaluation. So he is evaluating 
these people about the same as an employer would on a t 3 rpical 
factory benchwork .job using this as a guide. But we are not ask- 
ing him to evaluate factors that he is not qualified to evaluate. 



All of our clients come from the Division of Vocational 
Rehabilitation or from the Blind Ser\rLce Division. Now it does 
not seem on the face of it that any of these jobs could be done 
the blind. This in a sense is true. There are some people 
from the Blind Service Division who have had surgery or have 
just had recent cataract surgery with recovering vision and could 
do a limited amount of work with these transistors after they have 
been inspected, '’or m.-\i t.r mister ''•s to 

be marked with a little green magic marker to show that it is a 
reclaimed part. This is something a person with very little 
vision can do. All they have to do is pick it up and mark it 
across the top with the green marker. Vocational Rehabilitation 
sends people with all kinds of disabilities, orthopedic handicaps, 
respiratory illness, and cardiacs. We have a considerable number 
of mentally ill from our rehabilitation house program in Burlington, 
Montpelier. We are getting an increasing number of retarded who 
are sent to the shop simply tc try them out in the work setting. 
They may or may not have had very highly developed rehabilitation 
services in the past, but they are sent for this purpose. 



The reason we restrict intake to the Vocational Rehab- 
ilitation Agency is that the shop does not proidde any supporting 
services itself. We want to make sure, first of all, that these 
people do have a qualifying disability and secondly, that the 
shop experience, no matter vihere it occurs in the rehab process, 
is part of the rehrabilitation plan. 3o you see, the counselor 
then becomes an important part of the overall evaluation process. 

He knows what the shop has to offer. He knows the limitations of 
shop evaluators and he himself then becomes in a sense a part of 
the evaluation team. He does this by getting these periodic re- 
ports. Thej'’ are sens to him monthly. He comes into the shop at 
intervals and sits doim v/ith the shop manager and shop floor people 
and Mxth his skill and imov/ledge can determine whether or not his 
client is benefit-ting from the shop program. One of the big pro- 
blems we have at. the shop, and it may be a very undesirable sec- 
ondary dependency traits. We have not had the success we hoped 
in moving these people out of' the transitional emplojcnent of the 
shop into work experiences in the area. The counselors tend to 
get a client in the shop and bec£ se they are being paid a good 
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wape, leave then and ^or^et about then. They nay not try to move 
then .alcn^ for further evaluation or trainin", or actually into 
job placement. 7or some people it becomes very obvious after having 
worked with then for a long tine that they will have to be closed 
in an e:'Ctended shop setting. They vjell probably be permanent sheltered 
workshop employees. We are trying to hold this number do\m to a 
very small percentage. I think that is about all I have to offer 
and would welcome questions as a means of amplifying our program. 

QUESTIONS AND ANSWERS 

EGER?IAN: The question that I am going to ask, is that I wondered, 

is a particular skill which these people have that I have not seen 
elsewhere in any of the workshops that I have previously visited, 
and that is the capacity to perform highly sophisticated monitoring? 
Perhaps some of you who have had some more specific experience in 
the work evaluation programs, could answer. Pjre you aware of tasks 
that require only monitoring behaviors? The reason I bring this 
up is that, of course, monitoring behavior is an entire field of 
psychological perception in itself. When I was vjith an industrial 
research company, one of our projects was to find out why certain 
people are making errors on the large atomic reactor where they 
might have caused what is called a "critical incident" which can 
be anything from not noticing that a hot water pipe had burst to 
one where the vjhole thing just goes up and there is no one left 
around to tell you what had happened. If you have never had an 
opportunity to visit one of these installations, there are about 
70 dials and maybe about 20 meters and maybe about four records 
from an elaborate recorder that the monitor is e:q>ected to watch. 

What this job entails is just sitting there and periodically scan- 
ning all of these dials. It is very much like an airplane pilor. 

Once he gets the plane up in the air and sets it for automatic 
pilot, he just sits back and rests .and relaxes. But really that 
is not the end of it because all ••^f these dials have to monitored 
periodically and there are certain kinds of people soparently who 
are better monitors -- who are willing just to sit back and mon- 
itor; they do not have to act. This analogy, of course, is quite 
similaj:’ to people I have observed in your shop, where regardless 
of the severity of the physical disability, I am not sure I could 
say that with respect to the severity of the mental or emotional 
disability, but I seem to feel that there is a whold nev; avenue to 
a whold new approach to sequence of jobs to people who are in a 
workshop situation could undertake, ?jid this is the quality control 
type of operation monitoring. When I saw this operation, a fellow 
was sitting at a binocular microscope arid did nothing but let these 
little wafers pass underneath the microscope each time, hour after 
hour, after hour, and I guess you pointed out to me about lU or 
15 different kinds of things that he has to look for on each one 
of these little wafers for which he either accepts or rejects the 
wafer any single defect out of the lli. Have any of you had ex- 
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perience in this particular kind of work ?.s it misht be ccnducted 
in a workshop? As far as I know this is the only place in the 
world that does monitoring behavior so well. 
priLL: It is too bad that we have not developed this shop more 

professionally, with more professionally trained people and that 
we have not done more research into what we have found out, be- 
cause I am sure that we have found many interesting things about 
behavior of people that we are not aware of . We are simply aware 
of the fact that we can derive some benefit, the clients can de- 
rive some benefit by exposing them to a paid work experience. 

But there are many things that are going on at this shop that 
probably should be researched. 

AUDIENCE: I am a little bit concerned, or are you concerned, a- 

bout the fact that we seem to be confining your evaluation Per- 
sons who do this monitoring work and perhaps make decisions that 
they are not feasible for work or any work which obviously they 
might be doing. We have had other kinds of tasks used, closer 

HILL: Yes, one "'f our big problems has »-een too much^success with 

what we are doing and with the available facility, ^very week I 
talk to my shop manager to say we must get out and find some di 
ferant operations to get in here so we will have a broader scope 
of work, different tasks on which we can evaluate people. But lan 
keeps us so loaded Tfith work, we simply cannot get away from them. 
This is a point well taken, but we do not do any predicting. We 
do nr»t say that because the client cannot do this; he probably will 
not be able to do something else. All we are saying to the coun- 
selors is that he cannot do this particular job. 

AUDIENCE: Do you depend only on IBM quality control or do you nave 

your own? 4-u 

HILL: No, we have to maintain our own. If we do not, they win. 

AUDIENCE: How many of the workers are doing inspecting? ^ 

HILL: Well, probably about 60 to 6 S% of the shop are on some kind 

of inspection. With transistors, the same person that straightens 
the leads with the little pair of needle nose pliers, at the same 
time is inspecting the components that he is working on. Because 
he is working under an illuminated magnifying -lass, he is looking 
for these things as he works on the transistor. This is a combin- 
ation of manipulative skills and inspection at the same 
AUDIENCE: What are your e:qperiences with the mentally retarded. 

regarding monitoring? . ^ x t 4. 

HILL: Not too good. We had one who was just moderately retard 

that did do some inspection and did a fairly good job, but he 
could not discriminate at any given time. You could teach him to 
look for one defect and then for another defect and he could re^g- 
nize each defect in isolation, but as he had to do this on a hi^- 
speed basis during the day, he simply was forgetting it. 

AUDIENCE: How much practice did you give him to learn to monitor. 

HILL: Oh, probably four or five days of concentrated work, enough 



Hill 






so that we knew that with a lot. of instr’jiction he could recognize 
each single defect o He had no trouble at all. It was when he 
had to look at all lu of them at the same time, he would see one 
and forget really what he was looking for because he had to look 
for 30 many It was very obvious that this was not the thing for 
him to do. On the other hand, when he started working with tran» 
sistors where had to just pick up the transistor and straighten 
the lead and all he had to look for was to see if the lead was 
bent too flat to straighten or if it had some knicks in the side 
of it, he could remember this. And he a very satisfactory job. 
Interestingly enough, IBM who had never employed the mentally 
retarded before were so impressed with this performance that they 
asKed me to come up and survey their plant. We found at least a 
half dozen or a dozen stations in the IBM plant that would be very 
good fcr the mentally retarded and they have hired one on a pilot 
basis. Right now we are very anxious to see how he does. 

AUDIENCE’ Are you particularly interested in jobs for those people 
with limited or no use of their hands? Can*t these jobs require 
no use of the hands? 

HILLi We had one man, I think we have only had one. He had a 
prosthesis; just one hand and a prosthesis in a place of the other. 

He could do ii.spection, he could not do any manipulative jobs, but 
he could do inspection jobs. 

AUDIENCES The inspection jobs would not include any use of the 
hands at all? 

HILLi No. He had a hook, and he just pushed the work through with 
it. This job is set up with a tray on one side and these sub-straight 
all lined up in the tray. He just simply had to push a whole line 
of substrates along, one at a time, underneath the scope and they 
went over to ancthor tray. So he was able to do this without any 
difficulty. 

AUDIENCE:. Did he pull or mark the rejects? 

HILLs Nc, The rejects were pulled out and simply thrown into a 
rejection basket. 

AUDIENCES He could do this with his other hand? 

HILL; Yes. 

AUDIENCE; Who pulls them out? 

HILL; Well, the process can be stopped when he gets to the one 
that is bad. He can simply bring his hook up and push them back 
again and then push the reject of the end of the holding tray into 
the reject tray, wihn ths use of his arem and good hand. 

AUDIENCE. How about traumatic quads? 

HILLr I do not think they could do this without any hand function. 
AUDIENCE? What criterion did the Rehab office or counselors use 
to send people up to the workshop. 

HILL? They send them at different points in the rehabilitation 
process, Semetimes they will send clients over even before rehab 
really begins. They m?>y have already determined tentatively a vo- 
cational. objective for others. While they are waiting to get the 
client itito a training situatjon. they nay send them over for a 
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month cr six weeks ^ simply to test ability to work in a work assign- 
ment , They may send them over after they have come out of training, 
between the time training is finished and they go out on the job 
simply to see how they perform as workers » The counselors know 
that they have learned a skill and they, the clients, have been 
trained for a specific skiil, but they do not know how they work 
in an actual job^ 

AUDIENCES What percentage have been kept on as permanent workers? 
HILLS There has been a fairly small num*ber of clients that have 
actually been closed on the job that we know will be there as 
terminal shop workers. I think this is only about 10 per cent of 
the total work force o But we do have a great many that are tran- 
sistional and have been there a great deal longer than they should 
be. 

AUDIENCES Are there any specific characteristics that are recog- 
nized as being specific of this type of work? In other woris, are 
the counselors developing any ideas of what person will do better 
in this type of work? 

HILLj I do not think they have made this correlation, but I think 
generally speaking, they send them there simply to expose them to 
a paid work experience » They may even send a client to the shop 
that has shown in testing that he has very few skills for this 
type of work. They simply want to see how they can perform in a 
paid work experience. So it is used at anytime during the rehab- 
ilitation process, on a non -selective basis in regard to specific 
types of disability o 

AUDIENCES There is a mors general question I would like to raise 
and you brought it up in your talk. You said that perhaps in your 
own facility you are creating a certain amount of dependency on 
your facilities o In your opinion, what m.ight we dc to make these 
people independent in the sense they are independent because they 
are there working and getting a salary, etc. But should we be 
examining out programs and what not to determine if there are some 
things that we can dc to make them want tc leave us and these shops 
and find employment? It is a very sweet deal, so why leave it? 

So, what can we do? 

HILLs Well, I am. not sure. At least in our case, in our state, 
probably one of the biggest problems a counselor has is simply 
finding job opportunities. This has been traditionally hard in 
Vermont because there are very limited selective placement oppor- 
tunities in the state. Secondly, we have not had the co-operation 
from the employment service that we would have hoped for in this 
city. Here we have a shop with people who are doing fairly high 
level work. At the same time, in the city of Burlington, the em- 
ployment service has many unfilled job orders for factory bench 
workers. We have invited them down to the shop to see what we are 
doing- We say, "here we have people who are not unknowns to you. 
You have specific requests for specific types of workers and here 
we have them in the shop. And while we have not conscientiously 
trained them for factory benchwork, here they are doing it, and 
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doine a pood iob of it, in "inny coses, producing at rates from 50- 
ino per cent over established IBM norms." And yet «e still have 
these people stuck at ihe shop and not moving out. There ^re some 
hopeful sivr.E. I think IB” hos hired quite a few of them. One of 
t^e thLgs^hat may have happened is that this shop is i^eVa^ri-’ 
IBM's conscience. This was a very good way not to hire the hMdi 
capped in their own facility. They could simply bring their work 
down here and let ns do it. 

AUDIENCE: They do not hrve to heave any retirement plans / 

kind of fringe benefits or insurance for these people either, that 

is true isn’t it? 

HILL: You mean of their own. 

AUDIENCE: No, if they bring the work down to you. ^ 

HILL: Oh, no, that is very true. It is handled simply as a su - 
contract. Very true. I bo not think that this has been the thing 
?hft ha^kep; them from doing it. I think IBM has been very reluc- 
tant to hire the kind of people we have in the shop beers,, .,he:. 
-hl'sscrh'/ hr;- hr -1..'. ■■•'■•-y -c-.r. me-.:; ■ e 

who have great potential for growth in the comply. And they do 
not want to hire mhem simply as benchworkers and have them remain 

always as benchworkers. ^ j 

AUDIENCE: Would you say that you could help hire .he h^dicapped 

in industry developing this program more and more like the real 

world at work? •+ .?„ 

HTTL* Well, it is now, except that we do not want to turn it in- 
to an extended employment shop if we can possibly help it. We 
want to have turnover so that we will be moving the people along 
in the rehabilitation process. Instead of having them get stuck 
here, they would move out into competitive employment. We recog- 
nize the fact that many of them won’t be able to, but I resist 
the effort cn the part of our counselors to turn this into a ter- 
minal sheltered workshop. , 

AUDIENCE: In recent months we have had the experience with several 

clients who are in one phase of the program where workshop the 
whole staff felt that these were so multiply disabled that it looked 
like they were going to be in extended workshop employment for the 
rest of their lives . For some reason they got mad at us and left 
the shop and vient out and got their own jobs. 

HILL: How did you create this? ^ v, 

AUDIENCE: We are now In the process of trying to analyze why they 
got mad at us, to see how we can use it constructively. I do not 
know what the answer is. This happened and tells us something too. 
HILL: Well, one good thing that has come out of this is the people, 

after they have been there three or four months, begin to establish 
a standard of living that is commensurate with their earnings at 
the shop, but then as they begin to look around, they see that 
outside employment opportunity wil? offer them higher wages. They 
begin to get restless. They feel that they will not be able to 
advance within our shop and go onto better things, so they do begin 
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HILL-" Our services in Vermont are very limited. We send clients 
to out -of -St ate Rehab Centers. We do not have many facilities in 
our own state. We do have a medically oriented rehab center, but 
its services are limited largely to medical evaluation. In answer 
to your specific question, I think that we seek most rehab services 
elsewhere and use the shop largely to bring out what kind of a 
working person we have. It is also used to build motivation for 
work in those that seem to lack ai^ clear cut vocational goals or 
actual interest in employment c 

AUDIENCES Do you have to make any therapeutic changes in the client 
other than perhaps the motivation of one person? 

HlLLs Yes* I know many cases; for instance, when we first started, 
we had such a shortage of workers that we were pulling in some of 
severity of their disabilities. They had worked in the homebound 
program and demonstrated capacity for work. Then we brou^t them 
down to the shop. They really took off from this point. One lady, 

I remember, had enqphysema. She had never considered that she could 
leave the house. She just couldn't get out into the air. She 
has been at the shop not for almost two years and doing a very fine 
jcbr This is not so much planned therapeutic change as it is an 
accident of the actual work ejqperience, 

AUDIENCES Are there any staff members of the workshop who are charged 
with the responsibility of placement activities? 

HILLs No. 

AUDIENCES State en^)loyment service, our own VR counselors. We are 
toying with the idea of adding a placement person on our staff. 

Ri^t ROW we are hoping that we can interest the enqoloyment service 
group in doing more placement for us, rather than hiring someone 
ourselves. Have them detach a person and do placement in our own 
shop. 

AUDIENCES It seems your program is tied to IBM. 

HILLs Yes, ib is. In the Burlington area we have several large 
companies. General Electric has a manufacturing facility that would 
hire the same type of person for factory benchwork. There are two 
or three other small industries that are looking for this type of 
person . 

AUDIENCES Yes. 

HILL: You mean a mentally retarded person who is in the shop and 

decided because he can strai^ten leads on the transistors, thinks 
that he can become an electronic technician? I am not at all sure 
that we have run into the problem. So I really cannot answer your 
question. I suppose the counselor would have to discourage this 
kind of assuirqption of the part of the client. We have urged 
counselors to brief their clients very carefully before they come up 
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to look around for theins elves and begin to agitate for outside place- 
ment. This has happened, 

AUDIENCE ^ What’s your ratio of evaluators as such. This is shop 
production. We have a shop manager, we have two shop foremen. We 
have two assistant shop foremen, and we have one clerical person. 

This is a staff of six people who handle the total client load of 
75 to 80 people. So they are all involved in the production pro- 
cess and the evaluation is really a 1:^ -product. 

AUDIENCE: How do you remove the probably subjectivity in your 
evaluators rating? 

HILL: Simply by not asking them to do a very comprehensive job in 

evaluation. If you remember the factors that we are asking them 
to evaluate, I suppose that you could say, yes, there is a certain 
amount of subjectivity. For in%>tance, just taking out one element 
here; attitude toward sthff . Now this could certainly be very 
subjective. 

AUDIENCE: Attitudes toward staff; How do they judge acceptable 
attitudes they are after? 

HILL: They do not say whether it is acceptable or fair. They 
simply note what the client’s attitude was toward the people who 
are working in a supervisory capacity over them. 

AUDIENCE; Will you sum this all up? 

HILL: We do not reach conclusions and this is one of the things 
I do not want the staff to do because I do not feel that they are 
trained tc do it; I do not feel that the evaluation process, as 
we know it in this particular facility, has been geared to any 
summation. 

AUDIENCE? Who does this? 

HILL: The counselor takes the information as he receives it and 
usually in a talk with the shop manager or the foreman on the floor 
can reach his own conclusion about what they have observed. 

AUDIENCE; Each worker is e:q)osed to two or three at least or more 
supervisory personnel. You get a concensus. 

HILL: Yes, that is right. An evaluation report is always filled 

out. The shop manager, in conjunction with the floor people, sees 
that it is doned as a team rather than an individual evaluation. 

They staff each case. Another thing they do is to move the clients 
and expose them to as many job opportunities as they possibly can 
in the shrp. You see, when you are trying to run a shop and you 
are trying to make the shop pay for itself, (which is a very un- 
desirable thing in sheD.tered workshops), you have always to be 
conscious of your production goals. We have been able, as I have 
said here, to price our products sufficiently hi^ so that we can 
afford to carry a great many slow workers; so that we can afford 
to do some e:q)erimenting with them and carry them lot longer 
than we ordinarily would. 

AUDIENCE; What types of services are available for the rehab coun- 
selor to work with their clients and where does this fit into the 
program? How dees the counselor see the purpose of this in terms 
of ether available services.' 
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here; exactly why they are there, and that because they are working 
on this type of work doesn't necessarily mean they are going to be 
placed in this type of work or that if they do not do particularly 
well that means they cannot do anything very well. X think that 
they come reasonable well prepared for the experience that they are 
going to undergo. I do not know that its been a problem. 

AUDIENCE: Do you have any other jobs other than the quality con- 

trol such as dispatchers of the product? 

HILL: Yes. We have a recent job from IBI-l; a coil, the terminals 

of which have to be sealed with an epoxy resin. We have two or 
three people who do the epoxy mixing operation. The rest of them 
are simply putting this resin, after it is properly nixed and pre- 
pared, into these little contacts with a medical syringe, and seal 
the junction up. Now, I think what you ar6 asking, are these 
people doing a job that is not strictly concerned with the produc- 
tion of these things. Are they in a service job? 

AUDIENCE: Is the maintenance done by supervisory staff. 

HILL: There is not really that much maintenance. A very small 

shop in terms of floor space is only about U,000 square feet. We 
started out with about 2,000 and then expanded within the building 
to about U,000, but it is fairly small. It is not a big facility 
where there are a lot of maintenance or custodial jobs to be done. 

We could not provide full-time employment for them. 

AUDIENCE: What are the Vermont VR committments to your shop? You 
said they were not supporting you. 

HILL: None at all. No, I am on the State VR Division Staff. But 
the rest of these people are state employees, classified state em- 
ployees, but their salary is dependent on the income from the shop. 
And, as I say, the Legislature simply told us that we could continue 
to run the facility under state auspices. The Division actually 
runs it, but the staff jobs would only be in existence as long as 
there was money from contract income to support them and the shop 
operation as a whole. 

AUDIENCE: Why don't you make this a non-profit corporation and 
build your own state agency? 

HILL: I have been trying to create some interest in the Burlington 

area with a group that would take over the sponsorship of this. I 
do not think it is particularly appropriate for the state agency 
to be operating it at this point. 

AUDIENCE: Why? 

HILT.: It is inflexible. The State of Vermont has a very low salary 

structure. Because of this we are being held to their pay level 
and this makes it hard for us to get people to run the shop. And 
because we cannot depend on any support other than our contract 
commitments, we are very reluctant to expand the program in anv 
way. All we would need would be one slow period and the whole 
program would go down the drain. No, I think it should be sup- 
ported as a community activity. And subsidized, as any good shop 
should be. 
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AUDIENCE: What success have you had in training workers for mobility? 
HILL: In this shop, we make no effort to do it. 

AUDIENCE: Do these jobs, the workshop does, remain part of the IB! 

operation? That is, do they have the same work being done with 
their own labor force? 

HELL: Doing these? On same yes, on others, no. Some of these jobs 

have been farmed out not only to us, but to other ccmpanies in 
the area. 

AUDIENCE: Would you say that your workers are more productive than 

the regular workers? 

HELL: Well, let me say this. They exceed IB!'s own production 

standards on some operations by, 50-100 per cent. I would say a 
good per cent of our workers are exceeding IB!'s own production 
standards on certain operations. 

AUDIENCE: Is the workshop pay scale conqpetitive with IBl's? 

HILL: It is roughly competitive for this particular type of oper- 

ation. They do not have too many of these in their plant. These 
would be the lowest paying jobs in the IB! plant. And so, I think, 
this is one of the reasons they are trying to get them out of there. 
Our pay scale is competitive with other contractors in the area 
doing similar work for IB!, sometimes above. 

AUDIENCE: How is theirs established? Do you have any idea? 

HELL: We conduct time and motion studies. We have found that IBl's 

are quite liberal. They're lower than ours would have been if we 
had set our own. 

AUDIENCE: Do you think IB! studies are unrealistic? 

HILL: No, I do not think they are. I have watched them over there 

doing the same jobs and they just are not turning out the work that 
we are turning out. 

AUDIENCE: Are your referrals just from the Burlington area? 

HELL: No. These come from any counselor in the state. 

AUDIENCE: How doers the counselor learn about the client's progress 

besides from the written report? 

HELL: He does this on the phone if he is not immediately available.^ 

Counselors from the southern part of the state, and we are talking 
about distances of probably I 5 O-I 6 O miles, about once a week will 
call up and spend as much as an hour on the phone with the shop 
manager and the shop supervisoiy person discussing the report he 
just received what it means in texms of his client's perfor- 
mance there. 

AUDIENCE: Does the counselor come in regularly to see his client? 

HILL: While he is thereV Generally not, not if the client is from 
the southern part of the state, for instance, where 150 miles sep- 
arates them from the shop. 

AUDIENCE: Did you say the minimum wage is $1.25 «n hour? 

HILL: Yes. 

AUDIENCE: What happens if a person if found after two or three wedcs 
to be producing at their maxi mum $.6o an hour? 

HILL: We pay them $1.25. 

AUDIENCE: How long could this continue? 
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htt.t.« Oh we may keep them there as loiig as six „ 

ke“ them .8 much as three or four months Wins them 

a Mnimum, whiS is maybe twice as much as they are actually 

AUDIEHCE; VIhy do you pay them ''”opie*’ai^ living 

UTTT • W^ll in the first place because these peopx 

in residential facilities, because »®/° “f 
facilities for them. When they come to the *^®y “'"pj 

arrange for their own living accommodations with the help of 
l^elor. our feeling is that if we have • P®«°n thrt i^ 1^ 

thf rvarti^: ^rv^^^e 

s: 2: ^‘sSrs- 

^n is indicated and the on-the-job evaluation has served i 

aSot: Then the people that come there are not necessarily from 

Burlington? 

WTTT • Tffo thev arc from all over the state# • v. o 

PijtimCB:’ vnat peroenUge fin. Jobs, or the 

moved from the shop into employment. A great ”*°Maiiv of*^ 

the ^OT) into some other kind of training or evaluation. Many of 
tSLt^ve st^e^rit^t on at the shop, but I cannot tell you how 

Al^iIENCE: Going back to this other question. Since yo\ir shop is 

acquired by law as self-supporting^ . . 

TTiTT* Not bv law. By administrative edict. x /-/> 1 . 

AUDI0ICE: Since you pay the clients who earn on^ lowing th^" 

™ SLSns, that your good worker is supporting your poor 

mir'in a sense he is, but we have been able to P'^®® 

in such a way that our percentage of mark-\® ove _,,i,,gaY comes 
rate is pretty liberal. And generally speakxng, the subsidy comes 

^ r rit$:rtr 

d"S“ S.2SUT„-St 2 S ’ 

meT rather heart warming story about one 

in your city, which I think these other people might be quit 

^r“wf le«^"h?f'l.rticular building, ^ich i® 

Sles from the center of the city °f Burli^n, 

about four or five miles from the end of the bus route t a 
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down in that direction. To start with, we had to transport the clients ourselves 
from the shop up to the nearest bus stop and the bus then got them back into the 
city. So, I went to the bus company and explained our problem. As a result, the 
bus company very kindly extended their route down to the i^op, without asking me 
how much use it was going to get. They hoped that we would make as much use of 
it as possible and not depend on car pools for transportation because it was 
going to be costing them money if they did not get patronage. Initially they 
did not get enough usage, but continued to cooperate very nicely. When we had 
ni£^t shifts, they even said, "our last bus runs at 11:30, but we will come 
down at 1:00 at the end of this run to pick up your people, and on our way back 
to the terminal we will go all around the city of Burlington and drop them off 
at convenient stops." A very understanding group of people. 

AUDIENCE: Then you ask, why would anybody want to leave? 

HILL: Well, they have been very good to us. 

AUDIENCE: You talked about putting a counselor in the shop, who would be avail- 
able for counseling .... 

HIIiL: Well, speaking from the agency point of view, we cannot afford it rig^it 
now. And speaking from the shop point of view, I do not think, we could afford 
that either. I do not think the overhead would allow us to carry this much of 
a burden, to support the agency program. I do think we are going to have to do 
some of these things later on. This i^op grew like Topsy, and when you are a 
veiy busy people trying t>o run a production facility, you begin to think of the 
production aspects of the shop before you think of these other things. The 
further we go into it, the more we can see the need for many supporting services, 
and this is one of them. 

AUDlEN^: Do you do any other mental conditioning of the workers, such as 

leaflets, literature, films, or anything of this nature? 

HELL: Not at all. They are not there on off -hours, they come to work in the 
morning at 8:00 and are throu|^ at 4:30 in the afternoon. The shop is out in 
the country, and we do not have a lecture hall or a recreation area. Every 
single bit of space that is available is used for the shop production itself. 

All conditioning is provided by the counselor. In many cases a preliminary 
visit is arranged. Formal, structxired orientation is badly needed. 

KABL EGBIMAN: Dick has been very honest in describing his facilities. When I 
was up there this wedc going from the capital of Vermont ba<^ up to Bui'lingtoa, 
throui^ the lovely hillsides of New England, and we cane into a very nice 
residential area into what looked to me to be a ranch- type house, and maybe a 
double. I was looking around for the workshop, and Dick said, "well, there it is. 
We walked into a very attractive building, very clean, all on one floor, air- 
conditioning, and a twc-door loading dock; and everyone makes $1.25 an hour, and 
the civic groups in town cooperate. Why, that is a golden place to woiki 
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The Evaluation Department as the Pennsylvania Rehabili- 
tation Center is administratively located within the Counseling 
Service Unit, which includes Counseling, Case Management, and Vo- 
cational Evaluation. The goals of the Evaluation Department are 
to evaluate the vocational potential, to describe the individual 
vocationally, and to help the client achieve greater self-under- 
standing and a more realistic vocational outlook. 

Clients come to the Center from twelve district offices 
of the Pennsylvania Bureau of Rehabilitation, other states, other 
agencies, insurance companies, and as private in^viduals. V/e 
work with nearly every disability group except the blind. We are 
currently working on a program for the visually handicapped. There 
are several ways of classifying clients who come to the Pennsyl- 
vania Rehabilitation Center. One way would be disability. We 
have the orthopedic, the mentally retarded and the neuro -psychia- 
tric. There is 5;till another way of classifying clients, this is 
by functional classification according to vocational objective. 

We have then the physical restoration case, evaluation is of little 
concern. These indivLdauls come to the Center for Occupational 
Therapy, Speech Therapy, or possibly Physical Therapy, or combin- 
ations of these services. If no vocational objective is in view 
for physical restoration cases, then perhaps evaluation services 
may be sought, but usually these admissions return to former em- 
ployment. The training case has a vocational objective already 
established before coming to the Center. This is done by the re- 
ferring agency or the field counselor. The evaluation case, then, 
we are most concerned with. 



A BIT OF HISTORY 



When the Center first opened in 1959, a purely job try- 
out approach was utilized in the process of vocational evaluation. 
Job tryouts were selected in a relatively trial and error fashion 
with clients being placed in several vocational shops, usually ones 
of their choice for one or two week periods. The instructor in 
the course made judgments about the client's ability to do the 
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work and a selection of vocational objectives was made from the 
shops giving favorable reports. Some difficulties were encoun- 
tered i.’ith this method. First of all, it was very time consuming 
and there was little generalization to the overall potential of 
the individual. Observations were made by relatively untrained 
observers who had no way of determining whether or not a person 
would be able to complete a course. Many times it was found that 
work ability would plateau because of learning problems, even 
though the client may have done well initially in the program, 
wiuh the hiring of an occupational therapist, the emphasis was 
directed^ towards the xiork sample technique and the '»Tower system," 
One might say a xjcrk sample is a device in which certain critical 
aspects of a job are administered and performance measured in 
pother than actual job situation. For exan5)le, we can have sewing 
shoe repair, filing, mall sorting, mechanical assembly, etc. 

Some drawbacks are encountered in this method also. Many work 
s^les require considerable prior learning before an assessment 
of speed^or work quality can be made. Also, there is a limited 
number of work examples in which a person can perform because 
Ox physical space limitations, since there are many thousands of 
jobs to be saiT5)led. There are also geographical limitations. 

We serve people from throughout Pennsylvania and neighboring 
states. We could not possibly house the appropriate work sarmles 
for all jobs represented by people from all areas, 

THE PRESENT APPROACH 

Your attention is directed to level on the chart, 

"Process of Vocational Evaluation," that follows on the next page. 
Our present evaluation program begins at level one with the psy- 
chometric testing. This is completed the first week along with 
the orientation program to the Center activities. We gather 
information about reading ability, interests, general learning 
ability or ^telligency, personality, mechanical aptitude, general 
aptitudes, including verbal comprehension, perceptual speed, 
numerical reasoning, and spatial visualization. We also give a 
TOchaMcal aptitude test to males only. Training cases, as men- 
tioned earlier, progress only this far. We administer this test 
battery to the training cases in order’ to keep in touch with 
the population of individuals we are serving, and also, to generally 
coupare the abilitiAs with job requirements before they are placed 
tv course. More often than not, however, we honor the 

Dis..rict Counselor *s selection of a vocational objective and place 
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the individual in trianing even though he may be a marginal can- 
didate. The second week of testing begins with the General Apti- 
tude Test Battery. This is level two on your chart. This measure 
serves two important functions which are not served by the nrevious 
testing. First of all, it allows us to relate aptitude factors 
directly to predictions to job success by means of occupational 
aptitude patterns and also, secondly, it measures dexterity. Some 
of you probably know that this test was developed by the United 
States Employment Service and is used in the Pennsylvania State 
iijnployment offices. We have a special contract with the Pennsyl- 
vania State Employment Ser|Vice in order to administer the test. 

The test, itself, consists of a battery of twelve short, 
speeded sub-tests. These sub-tests give us scores on nine aptitude 
factors which include, general learning ability or intelligence, 
verbal aptitude, numerical aptitude, spatial aptitude, form per- 
ception, clerical perception, motor coordination, finger dexterity, 
and manual dexterity. These aptitude factors are considered, in 
various combinations of three. With cut-off scores at various 
levels, a person may meet, if his scores are high enou^, one or 
more of thirty -five occupational aptitude patterns, each of which 
suggest a variety of occupational areas in idiich a client's chances 
for success are about three out of four. This is a very important 
part of our testing program. After these standardized tests, the 
client is interviewed, test results discussed and an individual 
evaluation program planned, utilizing background, information ob- 
tained from the case record, medical information from former and. 
current medical examinations and any other vocationally signifi- 
cant information. 

At this point, we may have what we call a "GATE" referral 
"GATE' standing for General Aptitude Test Battery. Our program 
is designed, to meet the needs of the individual. We work with 
people who have a wide variety of abilities and. disabilities, the 
M.R. to the intelligent orthopedic disability. Knowing this, you 
can see that some individual would, not need evaluation further 
than psychometric testing. In fact, about of the eval.uation 
cases can be referred to the counselor for the selection of the 
appropriate vocational objective after the conqpletion of level 
two. The remaining cases continue for more intensive evaluation. 
These remaining cases need further evaluation when: 

1. Scores are poor and. no vocation is suggested. 

2. The nature of the involvement leads the counselor 
or the evaluator to suspect problems in making a 
vocational adjustment. 

3. A person's interests are ill defined. 

U. The counselor feels the need to establish the nature 
of the client's work habits, attitude, level of mo- 
tivation ability to use tools, ability to use the 
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prosthesis or an affected extremity, ability to 
follow insturctions of a written, oral, or demon- 
stration nature. 

5* It is felt necessary to Investigate capacity for 
learning simple motor skills, such as in the case 
of a cerebral palsy or hemiplegic individual. 

6. Any question arises from the client, counselor, e- 
valuator, or other staff member concerned with the 
client's vocational potential. 

V/e progress through the process only as far as is necessary to 
establish vocational goals. A great amount of responsibility is 
placed upon the counselor ^valuator team in this decision making 
process . 



At level three on your chart, you will notice perfor- 
mance meausres. These mrasures inclpde more highly individualized 
testing which not only involves dexterity tests, but also other 
achievement tests. The dexterity test nd^t include, Bennett Hand 
Tool Dexterity Test, Crawford Small Parts Dexterity Test, Ifinnesota 
Rate or lianipulation, or perhaps, the Purdue Pegboard. V/e may also 
at this level use the Wide Range Achievement Test, The Metropolitan 
Achievement Test, or the School and College Ability Test. Other 
measures such as the Msiitland -Graves Art Design Judgment measure 
and the Dennett Test for Mechanical Ccmprehension may be used. 

7or exanq>le, the client may be C(xisidering office machine repair, 
but his dexterity is questioned on the basis of General Aptitude 
Test Battery scores and further testing seems to be indicated. 
Dexterity testing can be done, along with observati(xi to investi- 
gate the feasibility of this particular client and the advisabil- 
ity of pursuing a course of training. 

At level four, we have the work activities. These in- 
volve a continued individualization and intensification of •Uie pro- 
cess. This stage includes a variety of work sample -like activities 
on which a client performs iftiile observations are made concerning 
his performance. Vb^'tianally significant, subjectively evaluated 
r.ictOTS are considered, such as the ability to follow instructions, 
to use tools, to use a prosthesis, his attitude and motivation, etc. 
Generally, we are evaluating the type of worker the client is. 

The work activities are selected according to questions about the 
client's vocational skills which do not appear answered by infor- 
mation gathered by other means. Exanples of work activities would 
include, parts of the Tower System, including the Clerical, Sewing, 
Drawing and lettering activities. In the cabinet making vjork sam- 
ple, we can determine how well the client measures, hc,v -sU he 
handles tools, whether he has an appropriate use of to«yls, how 
well he follows written and oral instruction, what the level of 
his independ ent judgment is, and can he learn fran his mistakes. 

On the gasoline engine, we can observe his use of mechanical tools 
and his familiarity with parts and operation. 
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You will notice the insertation on the chart labeled 
evaluation panel. This panel may appear at any point from there 
down in the process. The clients in evaluation are placed on a 
list with the person in evaluation longest on the top of the list. 
Cases are discussed weekly by counselors and evaluators in atten- 
dance. Problems and progress are reviewed in the evaluation pro- 
cess. This panel serves to facilitate case movement as well as 
to allow for particular counselors or evaluators to benefit from 
the experience of the group program planning and decision making. 

Shop tryouts are used on a very selective basis to pro- 
vide information not obtained throu^ previous means. They are 
used when: 

1. Other measures indicate prognosis of only marginal 
success . 

2. Behavior suggests problem areas. Ifany times the 
client will identify strongly with the more realistic 
job tryout setting and consquently perform more ^- 
propriately than he would in the evaluation unit 
setting. 

3. There are questions about the client's motivation or 
work habits. 

U. The person's Interest or degree of knowledge about 
the job is uncertain. 

5* As a reality test if the client is non -realistic and 
will consider nothing else until given a chance in 
a particular area in question. The evaluators try 
to ask specific questions about what the client can 
and cannot do in the particular area* 

Clinical observation is the final level listed on the 
evaluation process chart. Observations are iiq^ortant^ thoughout 
the evaluation program^ but careful observation on routine^ rep- 
etitive operations^ such as mechanical assembly and sorting often 
provides a clue to the type of activity a severely disabled indiv- 
idual might best seek. MEuiy times^ throu^ observations made^ 
selective placement recomnendations are established for sheltered 
workshqp^ homebound enqployment or work with a benevolent employer 
who may be willing to accept a particular limitation of a client^ 
still using the skills he has to offer. 

Not all persons have the same evaluation program. They 
do not spend the same amount of time in the process. The average 
time is approximately four weeks ^ with a minimum of two weeks and 
up to two or three months^ depending on the individual's needs. 

With about of our evaluee's we are able to establish a vocational 
objective after level two, another 60 to 60 % after levels three 
throu^ five, and the remainder have recommendations made after 
progressing throu^ the most individualized part of the process. 
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There has been a general streamlining of the evaluation 
process since the opening of the Center. We feel that we have a 
more efficient and a more effective program. This is part of an 
overall effort to give the most appropriate service to the most 
people per year. With the original job tryout approach to eval- 
uation^ about 100 to 150 clients were served per year . The work 
sanq)ling system allowed for the evaluation of 200 to 225 clients 
per year. The present system e^q>edites service to 550 to 6 OO 
clients per year. 7S% of the Center's admission are for evaluation 
services. We have a 6 to 8 month waiting list for evaluation, 
illustrating a demand for this particular type of service in 
Pennsylvania. 



THE REPORT 



* Within the Background Information section, factors sic- 

^ social history, if obtainable. In the TesflS^tSi^^ec- 
*'®Port test results in ranges and what the different 
mean. We indicate significant factors to which the SL 
In Sc ^ aptitude, clerica abiUty, or whatever 

s?gSfiSS? ^ocational^ 

subjectively evaluated factors such as: 

attitude, initiative, mann^S“n* 

«nWbmS S f independent judg- 

ThS’alS^mor? ““ prosthesis, or affected extremity.^ 

activity areas in which a person was tried 

sectSn irSotoblv^S®* eMorts. The Summary aSd ReconmendatiSs 
section is probably the most inqportant part of the reoort Hov.« 

^ draw together the signified inforiltion ab^t ^S^SienJ ef- 
fecting vocational potential. We sunaarize test reSltf ohLSl 

as°rMSultSf*'^fS* commonly reconmend^ 

as a result of vocation«a evaluation are: 



1 . 

2 . 

3 . 



Vocational training at the Center 
Training at^ other facilities. 

Selective placement, with skills already available. 
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U. Continued education^ such as finishing high school 
or going on to college. 

Sheltered workshop-work adjustment when a person 
could not enter the competitive labor market. 

We InqpUcltly attach probability of success to the var- 
ious things recomnended qualifying statements. This Is the 
major use of the behavioral observation section. For exuig>le^ 
this state nd^t appear In this section of the report. "The client 
may perform In the office machine repair area^ If his math and 
reading can be Improved. He would be best only as a helper In 
mechanical repairing trades with his present level of ability. 

The clerical areas are to be eliminated on the basis of Interest, 
but also on the basis of low scores In clerical perception and 
verbal aptitude." 
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In sunmary then, we generally consider four major areas 
about the client's personal vocational ability. 

1. Intelligence or academic ability. 

2. Dexterity or upper extremity functioning. 

3 . Personality factors, motivation, and desire for work. 
U. Disability, the functional limitations. 

The client's level of functioning in each of these four 
areas is judged against the frame of reference of occuaptional re- 
quirements such as found In the Dictionary of Occupational Titles. 
This would Include the professional, technical, service, skilled, 
semi-skilled, unskilled employments. Recosnendatlons are made 
with the appropriate qualifications. Our practice has been to 
recommend with qualifications, areas where there Is marg inal po- 
tential so that the client will at least be given an opportunity 
to try. Our reccmnsendatlons, of course, cannot bo made on the 
basis of interest alone. We simply point out to the client and 
to his counselor, areas in which he may be successful. This may 
be called vocational description. The counselor, in turn, assists 
the client in establishing a specific vocational objective. 

QUE3II0MS AND ANSWERS 

QUESTION: Do you use The General iptltude Test Battery on all 
clients? 

ANSWER: On all the clients that come for evaluation. That Is 
right. 

QUESTION: What ap^oxiaate miabara can you recoamend directly 
for placement? 
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ANSWER: About after the testing. Now this number is really 
a little lower than this now. There has been a tremendous shift 
in the type of population we have been getting. When I first 
came here about three years ago, we had a greater number of ortho- 
pedic disabilities, physical disabilities, and there has just been 
a tremendous shift to about 605? to neuro -psychiatric and mentally 
retarded. That is what we are handling now. So we are going to 
have to ad. just our evaluation program and we haven’t done this 
because of staff shortages, and just keeping up with the demand 
for evaluation services. 

QUESTION: What is the proportion of success that the Rehabilitation 
Center has in picking out a vocational objective for a student 
and getting them trained to the place as opposed to the success 
the field counselor has when he prescribes help in the training 
area? 

ANSWER: We have been approximately 755? successful with our eval- 
uations. This is not based on any real carefully controled study. 
But in general survey we found that about 755? of our people that 
we evaluate can be established in some sort of enqployment after 
training. Whereas, the field counselor sending the person in for 
training is only about 505? successful. 

QUESTION: Is the Level 1 testing done here in addition to Dis- 

trict testing? 

ANSWER: There are mai^, many tests the trainee gets before he gets 
through this program. In the district about 805? of the people 
who come to the Center get VBB's from the district so we don’t have 
to do that, in fact we don’t have a psychologist functioning as a 
psychologist on our staff. We do this other testing because we 
are most familiar with this testing, we are most comfortable with 
it. We can gather standard types of information about everyone 
because many times people coming in from the outside have a variety 
of tests that we are not - - .co familiar with or that really 
are not the same kind of tests, so we have no comparison between 
them. This is why we gather these standardized types of statistics 
on them. 

QUESTION: Do you get the WAIS sub -test scores with the client? 

ANSWER: Sometimes we do, but not in every case, however. We do 
have two people on our staff who do give the WAIS under the auspices 
of the psychologist. 

QUESTION: Who are your evaluators? 

ANSWER: The evaluators here at the Center are all Master’s Degree 
rehabilitation counselors, hopefully, with a background in psy- 
chology. When I first took over the department, we had two people. 
Then it was up to four. Then we had a real exodus to California 
and there was a lot of real recruiting went on at the 196U Conven- 
tion, when we lost a lot of people. Since then our salaries have 
come up a little bit, and we are able to attract more people; we 
are up to five people in the department including myself, and I 
caz*]^ a hal f case load. But they are all Master’s Degrees in 
Re^bilitation Counseling. They are all from Penn State", inciden- 
tally. We have an internship program with Penn State. 
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QUESTION: What do you do if you have a client and his scores do 
not indicate any reasonable vocational prognosis? 

ANSWER; We nave a relatively involved process for handling a 
case like this. If the scores don’t tell us too much about him, 
we will head for the work activities area to find out some of the 
vocationally significant material^ subjective or evaluative as I 
have mentioned. Generally find out what kind of a guy he is, get 
to know him a little bit, find out what his aspirations are, and 
if he is really an ‘M and he is not going to be able to handle too 
many types of technical or trade areas. We may, if he is aspiring 
to it, give him some tryouts in the technical or trades areas if 
there is any chance at all. If there isn’t too much of a chance, 
and he is more or less realistic, we may try him out in one cf our 
services areas. These are the areas of sewing machine operation, 
which is good for the girls, the laundry, the kitchen, and the do- 
mestic areas. And sometimes groundskeeping in the maintenance 
areas. But, other than that we are not too well equipped to handle 
a large proportion, at least not the large proportion of mentally 
retarded clients, that we are getting now. 

QUESTION: Do you have any norms you have developed here? 

ANSWER: We have a wealth of information in our files, but no one 
has had time because of this case service demand to sit down and 
actually do a study, and which with any kind of predictive study, 
is really what you are implying here. This is one of my great, 
neede I think, a real felt need that I have, to do this, to develop 
some other activities, especially for the mentally retarded. Per- 
haps s(xne sorting activities, we have some sorting now, but it is 
less than I think we should, really to adequate. But we Just don’t 
have, we have academic types of things to evaluate people on, in- 
stead of the more or less routine or repetitive operations. I 
think, we could learn a lot from visiting the workshop situations; 
because we are getting this rather large proportion of mentally 
retarded clients who have work adjustment problems too. 

QUESTION; Do you get enough feedback from the district office? 
ANSWER: Not really. We get a list of people, who have come 
throu^ the Center, and their Jobs. And so we could do some val- 
idation studies in this way to find out whether they are working 
or not. But the follow-up is not adequate either. We don’t get 
every case so that one could do a comprehensive type of follow-up 
study. 

QUESTION: Yesterday Dr. Cobb was reviewing some of the research 
that had been done and pointed out that the client -evaluator re- 
lationship, but I am sure it is in some way significant. I don’t 
know what kind of predictor variables you would want to get into, 
whether it would be a rating form or a questionalre type of thing 
or what. But I really don’t know what significance this is in 
our establishment here. We both have a counselor and an evaluator, 
who have a relationship with a particular client. And he m^ go 
to one with one type of story and to the other with another type 
of story^ We have found this in certain instances. I had one 
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man who would go to the counselor and simply cry his head off. I 
didn't see any of this behsTlor in the evaluation unit. He saw me 
as a different type of individual, perhaps it was because of the 
nature of the environment of the evaluation. Also, it wps not as 
easy to cry in my office because there were large windows as it was 
in the counselors* office where there was no access to this or per- 
haps other factors in the counselor -client relationship, but I do 
not have anything significant. 
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The Evaluation Department administers the 6.A.T.B. to every trainee referred 
for evaluation. If Occupational Aptitude Patterns are met on the G.A.T.B.) the 
trainee is referred directly to Counseling. If the trainee fails to meet such 
patterns, he is kept for a more extensive evaluation. Such evaluation includes 
the administration of a numher of performance tests as well as an arithmetic 
knowledge and information test. Work samples are also administered when feasible. 

The admlnlntratlon of performance and achievement tests gives the Evaluator 
some objective information on the trainee's abilities and disabilities, and 
enables him to make an evaluation which is not entirely subjective. 

The results of these performance and achievement tests are included in the 
Evaluator's rei>orts in order to give the Counselor some objective data on 
trainee performance. The following is a list of test descriptions that will be 
an aid to interpreting these test results. 
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MULTIPLE APTITUDE TESTS 



Publisher: CALIFORNIA TEST BUREAU 

Authors: David Segal and Evelyn Raskin 

Factor Groupings 

I. Verbal Comprehension 
Test 1. Word Meaning 
Test 2. Paragraph Meaning 
Test 3. Language Usage 

II. Perceptual Speed 

Test 3. Language Usage 
Test 4. Routine Clerical Facility 

III. Numerical Reasoning 

Test 5* Arithmetic Reasoning 
Test 6. Arithmetic Computation 

IV. Spatial Visualization 

Test 7. Applied Science and Mechanics 
Test 8. Spatial Relations - Two Dimensions 
Test 9. Spatial Relations - Three Dimensions 

General analysis of test items: 

Test 1. Word Meaning - Examinee is asked to select one of four words 

which most closely resembles the initial stimulus word in meaning. 
The test is overloaded with adjectives in preference to other 
parts of speech. 

Test 2. Paragraph Meaning - Examinee is asked to read a paragraph and 
answer questions of content and interpretation. Four or five 
of these eig^t paragraphs are loaded in favor of the social 
sciences. Previous reading in the social sciences will probably 
have a positive influence on the score. Many of the questions 
could be answered without even reading the paragraph. It 
can be inferred that the specialized reader is given an advantage. 

Test 3. Language Usage - The examinee is asked to identify the parts 
of the sentences which contain errors in grammar, spelling, 
punctuation, and capitalization. Most of the sentences contain 
one or more errors, although there are two sentences without 
any errors. This test is supposed to measure both verbal 
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comprehension and perceptual speed, but it gives no method of 
determining whether individual items are missed because of a 
lack of verbal ability or a lack of perceptual speed. This 
leaves us with a pair of muddy factors. 

Test 4 . Routine Clerical Facility - Measures the individual’s rate and 
accuracy in checking likenesses and differences in names and 
numbers. The authors call this a speed test rather than a 
power test. 

Test 5. Arithmetic Reasoning - Each item consists of a verbal descrip- 
tion which gives the essential facts of the problem followed by 
four possible answers. The examinee is a^< to indicate 
which of the four answers is correct. 

The problems involve the following mathematical functions: 

I 

1. Addition 5. Use of fomOa 

2. Subtraction 6. Substitute Number Systems 

3. Multiplication 7. Use of Scales 

4 . Division 8 * Interpretation of Procediires 

The following elements are used in these problems: 

A. Whole numbers E. Coonion currency 

B. Fractions F. Square Measure 

C. Decimals G. Cubic Measure 

D. Percentage H. Pounds and Ounces 

I. Mnear Measure 

Test 6. Arithmetic Computation - The examinee is asked to work out 

his answer to the problem and then indicate which of four given 

answers is correct. 

The functions and elements involved are as follows: 

1. Addition of whole numbers, fractions, and decimals. 

2. Subtraction of whole numbers, fractions, decimals. 

3. Division of whole numbers, fractions, and decimals. 

4. Multiplication of whole numbers, percentage, fraction, 
and decimals. 
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The functions and elements involved are as follows: (cont'd) 

5. Ratio and proportion. 

6. Identifying the relationship of one number to another. 

7. Changing written verbal numbers to symbols. 

Test 7- Applied Science and Mechanics - A series of multiple choice 

questions on the nature of fluids, gases, and machines as applied 
to rej.atively familiar situations. Prior learning through 
science courses of mechanical experience will tend to elevate 
the score. This tends to measure the quality and effectiveness 
of previous exposure to content of this nature. This may give 
us, at the most, a crude index of preparedness for future 
learning of this type. 

Test 8. Spatial Relations - Two Dimensional 

A series of multiple choice items in which the completed figure 
is given the e x a mi nee and his task is to select the appropriate 
cluster of two dimensional parts which, when assembled, form 
a figure congruent to the original completed figure. 

Test 9 . Spatial Relations - Three Dimensions 

A series of multiple choice items in which the completed three 
dimensional figure is given and the examinee is asked to select 
the one of four two-dimensional patterns which, when con- 
structed, would form the completed figure. 

Critical Review 

One of the worst criticisms of this battery is that there is no al- 
ternative form for retest. This tends to make the test an all or nothing 
situation. 

The greatest claim to validity is the correlation between test scores 
and school grades. However, these correlations are disappointingly low. 

For Instance, the correlation, for males, between Applied Science and 
Mechanics and Hi^ School Physics is only .28. 

Internal consistency was emphasized at a sacrifice to pure factors. 
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APTITUDE TEST 

GEMERAL APTITUDE TEST BATTEBY ( G.A.T.B . ) 

The G.A.T.B. contains a limited number of diversified tests which 
can be given to everyone and which can be linked together in various com- 
binations to predict success in different situations. It consists of 8 
pencil-paper and 4 apparatus tests used to measure 9 distinct factors. 

The factors covered by the G.A.T.B. are as follows; 

G - UffTELLIGENCE ; General learning ability. The ability to "catch 
on" and understand instructions and underlying principles. Ability 
to reason and make judgments. Closely related to doing well in 
school. 

V - VERBAL APTITUDE : Ability to understand meanings of words and ideas 
associated with them^ and to use them effectively. To comprehend 
language, to xinderstand relationships between words, and to under- 
stand meanings of whole sentences and paragraphs. To present 
Information or ideas clearly. 

N - NUMERICAL APTITUDE ; Ability to perform arithmetic operations 
quickly and accurately. 

S - SPATIAL APTITUDE ; Ability to comprehend forms in space and under- 
stand relationships of plane and solid objects. May be used In 
such tasks as blueprint reading and In solving geometry problems. 
Frequently described as the ability to "visualize" objects of 2 or 
3 dimensions, or think visually of geometric forms. 

P - FORM PERCEPTION ; Ability to perceive pertinent detail in objects 
or In pictorial or graphic material. To make visual comparisons 
and discriminations and see slight differences In shapes and shadings 
of figures and widths and lengths of lines. 

Q - CLERICAL PERCEPTION ! Ability to perceive pertinent detail in verbal 
or tabular material. To observe differences in copy, to proofread 
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Q - CUmCAL lERCEPTiait : (cont'd) 

voids and nusberS} and to avoid perceptual errors in arithaetic 
cosqputation. 

K - MOIOR COORDIHATKai ! Ability to coordinate eyes and hands or fingers 
rapidly and accurately in making precise movements with speed. 
Ability to make a movement response accurately and quickly. 

F - FIMCER PE3CEERITr ! Ability to move the finders and manipulate small 
objects with the fingers rapidly or accurately. 

M - MAMOAL DEXIERIir t Ability to move the hands easily and skillfully. 

To voxk with the hands in placing and turning siotions. 

Success on the GJ^.T.B. is hic^ily dependent upon one's speed; the 
working time for paper-and-pencil subtests being close to 6 minutes each. 

The raw scores on the 9 factors are converted into standard scores with a 
Mean of 100 and a S.D. of 20. In general, the G.A.T.B. has proved to be 
one of the most successful multiple aptitude batteries in current use. 
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BEHK2TT HAID-TOOL D1 






Un TEST: 



The Hand-Tool Dexterity Test is designed to provide a measure of 
proficiency in using ordinary mechanics' tools. Althou^ performance is 
undoubtedly influenced by the subject's past eiperience in handling tools, 
the test was constructed so as to maximize the role of manipulative skill 
rather than mechanical information and/or "intelligence". The task is 
simply to remove all the nuts and bolts (of different sizes) from the left 
uprli^t and mount them on the ri|^t upright in a prescribed sequence. The 
score is the total time required to congdete this task. 

This test correlates ,U6 with performance of machinists. 
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un TEST 






The g— n Barts Dexterity Test is s pexfomsnce test designed to 
measure fine eye-hand coordination. In Bart I of this test, the subject 
uses tveezers to insert pins into close-fitting holes, and then places a 

collar over each pin. In Bart II, small screws are placed in threaded 
holes and screwed down with a screwdriver. The score is the time required 
to complete each part. Split-half reliability co-efficients between .80 and 
nre reported for the two parts of this test. Despite the apparent 
similarity of the functions required by Barts I and II, the correlations 
between the 2 parts ranged from .10 to .50 in several industrial and hi|^ 
school samples, with a median correlation of .42. 



MOHESOIA BATE ^ MABIBDLATiaE TEST ; 

The Minnesota Bate of Manipulction Test is designed to measure native 
speed capacity of eyes with hands and fingers in shop and office occiqpations, 
where it is necessary to execute quick movements in the handling of simple 
tools and production materials without the necessity of selecting different 
shapes and sizes. In a sense this test measures relatively simple muscular 
reaction to visual stimuli. Precision of eye-hand coordination is not an 
iiqportant factor , but the speed of gross arm and hand movements is significant. 

The test consists of a board containing 60 circular holes into which 
6o cylindrical blocks are to be placed (Bart l). In the second part of 
this test, each block is removed from the board, turned over with the other 
hand, and returned to its hole. The score is the amount of seconds it 
takes to do the "jdacing" or "ttiming" test four times. A correlation 
of only .57 was found between the "placing" and "turning" parts of this test. 
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PORDWE HBGBQAHD ; 

The Purdue Pegboard is a speed test idiich consists of Inserting 
pins into iwn holes, first with the rig^t hand, then with the left, and 
finally with both hands together* In another part of the test, pins, 
collars, and washers are assembled in each hole by using both hands sinul- 
taneously. The subject is given three trials on each part of the test. 

The score is the number of parts properly placed during a prescribed time 
limit. 

This test is said to provide a measure of two '^rp^s of activity, 
one requiring gross movements of fingers, bands, and ams, and the other 
involving **tlp of the finger" dexterity needed in small assembly work. The 
Asseiid>ly Test would be the most accurate measure of the latter actlvi'ty. 

ACHIEVEME3IT TE8T 

TIEDimH ARITHMETICAL CTOWIZPgE AM) IMFOliHkTiaH TEST ! 

This test is an individual and/or group test idiich measures objectively 
one's ability to respond to arithmetical problems involving a wide 
varie'ty of applications of arithmetic fundamentals. It can be used as: 

1. A survey test of arithmetical knowledge and information for 
grades 7 through 13. 

2. An instrument for the selection and classification of applicants 
for jobs or positions in business and industxy in which the 
knowledge and application of arithmetic fundamentals are inqportant. 

The Tledeman consists of 90 items, divided into 13 groups which 
sample the areas of addition, subtraction, multiplication, division, problem 
solving, and percentage. 
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